[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:17:03 dist-test-slave-6k22 krb5kdc[5371](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:17:03 dist-test-slave-6k22 krb5kdc[5371](info): set up 2 sockets
Aug 09 20:17:03 dist-test-slave-6k22 krb5kdc[5371](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 20:17:04.280594 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.007s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:17:04 dist-test-slave-6k22 krb5kdc[5371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:17:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:17:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:17:04.460815 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:36397
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:45847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:36397
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:04.811012 5386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:04.811378 5386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:04.811611 5386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:04.818652 5386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:17:04.818761 5386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:04.818882 5386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:04.819020 5386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:17:04.819089 5386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:17:04.829425 5386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:45847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:36397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:36397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:04.831344 5386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:04.833525 5386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:04.852833 5391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:04.853559 5393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:04.853931 5395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:04.854624 5386 server_base.cc:1034] running on GCE node
I20260809 20:17:04.855795 5386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:04.857875 5386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:04.859124 5386 hybrid_clock.cc:648] HybridClock initialized: now 1786306624859047 us; error 91 us; skew 500 ppm
I20260809 20:17:04.859715 5386 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:04 dist-test-slave-6k22 krb5kdc[5371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306624, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:04.868412 5386 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:17:04.870833 5386 webserver.cc:460] Webserver started at http://127.5.61.62:38187/ using document root <none> and password file <none>
I20260809 20:17:04.871999 5386 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:04.872167 5386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:04.872645 5386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:04.876716 5386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "f582d0e762974638a1a7f89e79c27872"
format_stamp: "Formatted at 2026-08-09 20:17:04 on dist-test-slave-6k22"
server_key: "d89284601577b1b66e344ac946528048"
server_key_iv: "f914374d9571c281aa9c749f2a9dfc3c"
server_key_version: "encryptionkey@0"
I20260809 20:17:04.877517 5386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "f582d0e762974638a1a7f89e79c27872"
format_stamp: "Formatted at 2026-08-09 20:17:04 on dist-test-slave-6k22"
server_key: "d89284601577b1b66e344ac946528048"
server_key_iv: "f914374d9571c281aa9c749f2a9dfc3c"
server_key_version: "encryptionkey@0"
I20260809 20:17:04.885353 5386 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260809 20:17:04.890479 5404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:04.892409 5386 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260809 20:17:04.892688 5386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "f582d0e762974638a1a7f89e79c27872"
format_stamp: "Formatted at 2026-08-09 20:17:04 on dist-test-slave-6k22"
server_key: "d89284601577b1b66e344ac946528048"
server_key_iv: "f914374d9571c281aa9c749f2a9dfc3c"
server_key_version: "encryptionkey@0"
I20260809 20:17:04.892896 5386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:04.908034 5386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:04.913468 5386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:04.913844 5386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:04.936822 5386 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:36397
I20260809 20:17:04.936866 5467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:36397 every 8 connection(s)
I20260809 20:17:04.939155 5386 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:17:04.946007 5468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:04.948112 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 5386
I20260809 20:17:04.948645 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:17:04.952026 5364 external_mini_cluster.cc:1428] Setting key f2b8ae4a3f5d9b9c441e60e36c78aa62
I20260809 20:17:04.965142 5468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f582d0e762974638a1a7f89e79c27872: Bootstrap starting.
I20260809 20:17:04.970307 5468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f582d0e762974638a1a7f89e79c27872: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:04.972395 5468 log.cc:824] T 00000000000000000000000000000000 P f582d0e762974638a1a7f89e79c27872: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:04.976480 5468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f582d0e762974638a1a7f89e79c27872: No bootstrap required, opened a new log
I20260809 20:17:04.985214 5468 raft_consensus.cc:348] T 00000000000000000000000000000000 P f582d0e762974638a1a7f89e79c27872 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f582d0e762974638a1a7f89e79c27872" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 36397 } }
I20260809 20:17:04.985649 5468 raft_consensus.cc:374] T 00000000000000000000000000000000 P f582d0e762974638a1a7f89e79c27872 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:04.985834 5468 raft_consensus.cc:729] T 00000000000000000000000000000000 P f582d0e762974638a1a7f89e79c27872 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f582d0e762974638a1a7f89e79c27872, State: Initialized, Role: FOLLOWER
I20260809 20:17:04.986824 5468 consensus_queue.cc:260] T 00000000000000000000000000000000 P f582d0e762974638a1a7f89e79c27872 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f582d0e762974638a1a7f89e79c27872" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 36397 } }
I20260809 20:17:04.987138 5468 raft_consensus.cc:388] T 00000000000000000000000000000000 P f582d0e762974638a1a7f89e79c27872 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:17:04.987425 5468 raft_consensus.cc:482] T 00000000000000000000000000000000 P f582d0e762974638a1a7f89e79c27872 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:17:04.987658 5468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f582d0e762974638a1a7f89e79c27872 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 20:17:04 dist-test-slave-6k22 krb5kdc[5371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:04.990332 5468 raft_consensus.cc:504] T 00000000000000000000000000000000 P f582d0e762974638a1a7f89e79c27872 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f582d0e762974638a1a7f89e79c27872" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 36397 } }
I20260809 20:17:04.991369 5468 leader_election.cc:302] T 00000000000000000000000000000000 P f582d0e762974638a1a7f89e79c27872 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f582d0e762974638a1a7f89e79c27872; no voters:
I20260809 20:17:04.992345 5468 leader_election.cc:288] T 00000000000000000000000000000000 P f582d0e762974638a1a7f89e79c27872 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:17:04.993011 5474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f582d0e762974638a1a7f89e79c27872 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:04.994151 5474 raft_consensus.cc:686] T 00000000000000000000000000000000 P f582d0e762974638a1a7f89e79c27872 [term 1 LEADER]: Becoming Leader. State: Replica: f582d0e762974638a1a7f89e79c27872, State: Running, Role: LEADER
I20260809 20:17:04.995075 5474 consensus_queue.cc:237] T 00000000000000000000000000000000 P f582d0e762974638a1a7f89e79c27872 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f582d0e762974638a1a7f89e79c27872" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 36397 } }
I20260809 20:17:04.995721 5468 sys_catalog.cc:564] T 00000000000000000000000000000000 P f582d0e762974638a1a7f89e79c27872 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:17:05.004102 5471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:04.958970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39530 (local address 127.5.61.62:36397)
0809 20:17:04.960209 (+ 1239us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:04.960234 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:04.964141 (+ 3907us) server_negotiation.cc:407] Connection header received
0809 20:17:04.974260 (+ 10119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:04.974315 (+ 55us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:04.974804 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:04.975240 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:04.978966 (+ 3726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:04.980550 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:04.982408 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:04.983186 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:04.983532 (+ 346us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:04.990281 (+ 6749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:04.990347 (+ 66us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:04.990379 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:04.990420 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:04.995474 (+ 5054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:04.996804 (+ 1330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:04.996814 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:04.996822 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:04.997194 (+ 372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:04.997977 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:04.997982 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:04.997985 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:04.999111 (+ 1126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:04.999692 (+ 581us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:05.001035 (+ 1343us) server_negotiation.cc:299] Negotiation successful
0809 20:17:05.001669 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":757,"thread_start_us":389,"threads_started":1}
I20260809 20:17:05.006605 5475 sys_catalog.cc:455] T 00000000000000000000000000000000 P f582d0e762974638a1a7f89e79c27872 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f582d0e762974638a1a7f89e79c27872" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f582d0e762974638a1a7f89e79c27872" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 36397 } } }
I20260809 20:17:05.007120 5475 sys_catalog.cc:458] T 00000000000000000000000000000000 P f582d0e762974638a1a7f89e79c27872 [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:05.008985 5476 sys_catalog.cc:455] T 00000000000000000000000000000000 P f582d0e762974638a1a7f89e79c27872 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f582d0e762974638a1a7f89e79c27872. Latest consensus state: current_term: 1 leader_uuid: "f582d0e762974638a1a7f89e79c27872" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f582d0e762974638a1a7f89e79c27872" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 36397 } } }
I20260809 20:17:05.009397 5476 sys_catalog.cc:458] T 00000000000000000000000000000000 P f582d0e762974638a1a7f89e79c27872 [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:05.014694 5481 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:17:05.023720 5481 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:17:05.037985 5481 catalog_manager.cc:1347] Generated new cluster ID: d5cee48f84004fe3806ef98683036156
I20260809 20:17:05.038144 5481 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:17:05.053313 5481 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:17:05.054811 5481 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:17:05.061750 5481 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f582d0e762974638a1a7f89e79c27872: Generated new TSK 0
I20260809 20:17:05.062685 5481 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.61.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
I20260809 20:17:05.117116 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:36397
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:45847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:05.453331 5502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:05.453581 5502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:05.453729 5502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:05.461093 5502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:05.461216 5502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:05.461387 5502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:17:05.471258 5502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:45847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:36397
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:05.472576 5502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:05.473824 5502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:05.487952 5507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:05.487926 5510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:05.488030 5508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:05.488837 5502 server_base.cc:1034] running on GCE node
I20260809 20:17:05.489403 5502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:05.490228 5502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:05.491420 5502 hybrid_clock.cc:648] HybridClock initialized: now 1786306625491390 us; error 27 us; skew 500 ppm
I20260809 20:17:05.491786 5502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:05 dist-test-slave-6k22 krb5kdc[5371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306625, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:05.495070 5502 init.cc:377] Logged in from keytab as kudu/127.5.61.1@KRBTEST.COM (short username kudu)
I20260809 20:17:05.496606 5502 webserver.cc:460] Webserver started at http://127.5.61.1:33455/ using document root <none> and password file <none>
I20260809 20:17:05.497301 5502 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:05.497452 5502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:05.497785 5502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:05.500777 5502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "5ac85857b9404c23ab4841cdd9a59c68"
format_stamp: "Formatted at 2026-08-09 20:17:05 on dist-test-slave-6k22"
server_key: "e5f6b6451b7c4207fb95fd92d216f04e"
server_key_iv: "5335fb3d5acbe282032fdffabbc3314e"
server_key_version: "encryptionkey@0"
I20260809 20:17:05.501513 5502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "5ac85857b9404c23ab4841cdd9a59c68"
format_stamp: "Formatted at 2026-08-09 20:17:05 on dist-test-slave-6k22"
server_key: "e5f6b6451b7c4207fb95fd92d216f04e"
server_key_iv: "5335fb3d5acbe282032fdffabbc3314e"
server_key_version: "encryptionkey@0"
I20260809 20:17:05.506500 5502 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 20:17:05.509845 5519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:05.511139 5502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 20:17:05.511317 5502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "5ac85857b9404c23ab4841cdd9a59c68"
format_stamp: "Formatted at 2026-08-09 20:17:05 on dist-test-slave-6k22"
server_key: "e5f6b6451b7c4207fb95fd92d216f04e"
server_key_iv: "5335fb3d5acbe282032fdffabbc3314e"
server_key_version: "encryptionkey@0"
I20260809 20:17:05.511524 5502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:05.533778 5502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:05.537695 5502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:05.537966 5502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:05.539026 5502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:05.540717 5502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:05.540834 5502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:05.541052 5502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:05.541132 5502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:05.576325 5502 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:33443
I20260809 20:17:05.576414 5651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:33443 every 8 connection(s)
I20260809 20:17:05.577627 5502 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:17:05.584447 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 5502
I20260809 20:17:05.584676 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:17:05.585333 5364 external_mini_cluster.cc:1428] Setting key cfdc9c6f3156682dd1bfd7b8f83cda64
Aug 09 20:17:05 dist-test-slave-6k22 krb5kdc[5371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306625, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:05.599381 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:05.581016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:47865 (local address 127.5.61.62:36397)
0809 20:17:05.581516 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:05.581521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:05.582379 (+ 858us) server_negotiation.cc:407] Connection header received
0809 20:17:05.584353 (+ 1974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:05.584358 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:05.584453 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:05.584576 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:05.586297 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:05.587175 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:05.588815 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:05.589227 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:05.589372 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:05.593058 (+ 3686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:05.593067 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:05.593074 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:05.593118 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:05.595873 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:05.596815 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:05.596819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:05.596821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:05.596939 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:05.597438 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:05.597441 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:05.597443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:05.597810 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:05.597933 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:05.598974 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0809 20:17:05.599140 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":283,"threads_started":1}
I20260809 20:17:05.600989 5657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:05.581455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:36397 (local address 127.5.61.1:47865)
0809 20:17:05.582217 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:05.582257 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:05.584067 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:05.584853 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:05.584861 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:05.585419 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:05.586088 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:05.586121 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:05.587352 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:05.587360 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:05.588665 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:05.588679 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:05.589565 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:05.589574 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:05.589788 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:05.590463 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:05.590484 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:05.592821 (+ 2337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:05.596004 (+ 3183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:05.596014 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:05.596032 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:05.596705 (+ 673us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:05.597055 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:05.597063 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:05.597070 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:05.597337 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:05.597952 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:05.597963 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:05.598421 (+ 458us) client_negotiation.cc:769] Sending connection context
0809 20:17:05.598817 (+ 396us) client_negotiation.cc:240] Negotiation successful
0809 20:17:05.599412 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":308,"threads_started":1}
I20260809 20:17:05.602747 5652 heartbeater.cc:347] Connected to a master server at 127.5.61.62:36397
I20260809 20:17:05.603154 5652 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.5.61.2@KRBTEST.COM; defaulting to no policy
I20260809 20:17:05.604081 5652 heartbeater.cc:511] Master 127.5.61.62:36397 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.2@KRBTEST.COM" created.
I20260809 20:17:05.607805 5427 ts_manager.cc:194] Registered new tserver with Master: 5ac85857b9404c23ab4841cdd9a59c68 (127.5.61.1:33443)
I20260809 20:17:05.609786 5427 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.1@KRBTEST.COM'} at 127.5.61.1:47865
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
I20260809 20:17:05.627424 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:36397
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:45847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:05.968484 5662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:05.968796 5662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:05.968904 5662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:05.976536 5662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:05.976648 5662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:05.976773 5662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:17:05.986828 5662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:45847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:36397
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:05.988132 5662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:05.989408 5662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:06.002938 5667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:06.002892 5668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:06.002957 5672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:06.004000 5662 server_base.cc:1034] running on GCE node
I20260809 20:17:06.004611 5662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:06.005584 5662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:06.006821 5662 hybrid_clock.cc:648] HybridClock initialized: now 1786306626006764 us; error 48 us; skew 500 ppm
I20260809 20:17:06.007130 5662 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:06 dist-test-slave-6k22 krb5kdc[5371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306626, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:06.010456 5662 init.cc:377] Logged in from keytab as kudu/127.5.61.2@KRBTEST.COM (short username kudu)
I20260809 20:17:06.011965 5662 webserver.cc:460] Webserver started at http://127.5.61.2:34087/ using document root <none> and password file <none>
I20260809 20:17:06.012687 5662 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:06.012837 5662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:06.013175 5662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:06.016444 5662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "1d20e3fb51fd453288ffb7c9268d667b"
format_stamp: "Formatted at 2026-08-09 20:17:06 on dist-test-slave-6k22"
server_key: "1e5a5ab5a652e17b3c8a9eaaddf28010"
server_key_iv: "7608ed4c58de46192c74c8083d176a30"
server_key_version: "encryptionkey@0"
I20260809 20:17:06.017216 5662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "1d20e3fb51fd453288ffb7c9268d667b"
format_stamp: "Formatted at 2026-08-09 20:17:06 on dist-test-slave-6k22"
server_key: "1e5a5ab5a652e17b3c8a9eaaddf28010"
server_key_iv: "7608ed4c58de46192c74c8083d176a30"
server_key_version: "encryptionkey@0"
I20260809 20:17:06.022778 5662 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 20:17:06.026554 5679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:06.027895 5662 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 20:17:06.028075 5662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "1d20e3fb51fd453288ffb7c9268d667b"
format_stamp: "Formatted at 2026-08-09 20:17:06 on dist-test-slave-6k22"
server_key: "1e5a5ab5a652e17b3c8a9eaaddf28010"
server_key_iv: "7608ed4c58de46192c74c8083d176a30"
server_key_version: "encryptionkey@0"
I20260809 20:17:06.028277 5662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:06.042873 5662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:06.047163 5662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:06.047487 5662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:06.048496 5662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:06.049889 5662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:06.049990 5662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:06.050140 5662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:06.050220 5662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:06.086442 5662 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:44045
I20260809 20:17:06.086551 5809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:44045 every 8 connection(s)
I20260809 20:17:06.087765 5662 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:17:06.094161 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 5662
I20260809 20:17:06.094378 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:17:06.094816 5364 external_mini_cluster.cc:1428] Setting key 3470709f8c78cb5116a0b480f7d8aa3a
Aug 09 20:17:06 dist-test-slave-6k22 krb5kdc[5371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306626, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:06.111588 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:06.090417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:41973 (local address 127.5.61.62:36397)
0809 20:17:06.090601 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:06.090606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:06.091829 (+ 1223us) server_negotiation.cc:407] Connection header received
0809 20:17:06.093641 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:06.093646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:06.093785 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:06.093910 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:06.095536 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:06.096558 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:06.099133 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:06.099645 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:06.099746 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:06.104325 (+ 4579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:06.104333 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:06.104341 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:06.104386 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:06.108149 (+ 3763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:06.108881 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:06.108890 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:06.108896 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:06.109067 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:06.109497 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:06.109505 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:06.109511 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:06.109900 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:06.110049 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:06.110840 (+ 791us) server_negotiation.cc:299] Negotiation successful
0809 20:17:06.111096 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 20:17:06.112310 5814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:06.090872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:36397 (local address 127.5.61.2:41973)
0809 20:17:06.091661 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:06.091717 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:06.093387 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:06.094108 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:06.094116 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:06.094744 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:06.095368 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:06.095385 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:06.096735 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:06.096742 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:06.098978 (+ 2236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:06.098998 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:06.099910 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:06.099920 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:06.100264 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:06.101097 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:06.101132 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:06.104102 (+ 2970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:06.108294 (+ 4192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:06.108300 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:06.108313 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:06.108751 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:06.109184 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:06.109187 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:06.109189 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:06.109386 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:06.110028 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:06.110034 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:06.110396 (+ 362us) client_negotiation.cc:769] Sending connection context
0809 20:17:06.110707 (+ 311us) client_negotiation.cc:240] Negotiation successful
0809 20:17:06.111115 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":308,"threads_started":1}
I20260809 20:17:06.113865 5810 heartbeater.cc:347] Connected to a master server at 127.5.61.62:36397
I20260809 20:17:06.114207 5810 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.5.61.3@KRBTEST.COM; defaulting to no policy
I20260809 20:17:06.115173 5810 heartbeater.cc:511] Master 127.5.61.62:36397 requested a full tablet report, sending...
I20260809 20:17:06.116966 5427 ts_manager.cc:194] Registered new tserver with Master: 1d20e3fb51fd453288ffb7c9268d667b (127.5.61.2:44045)
I20260809 20:17:06.117830 5427 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.2@KRBTEST.COM'} at 127.5.61.2:41973
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
I20260809 20:17:06.138968 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:36397
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:45847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:06.468152 5817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:06.468443 5817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:06.468626 5817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:06.476136 5817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:06.476291 5817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:06.476529 5817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:17:06.486680 5817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:45847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:36397
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:06.487939 5817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:06.489248 5817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:06.503155 5823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:06.503123 5824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:06.503197 5826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:06.504407 5817 server_base.cc:1034] running on GCE node
I20260809 20:17:06.504966 5817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:06.505856 5817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:06.507053 5817 hybrid_clock.cc:648] HybridClock initialized: now 1786306626507011 us; error 39 us; skew 500 ppm
I20260809 20:17:06.507354 5817 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:06 dist-test-slave-6k22 krb5kdc[5371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306626, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:06.510452 5817 init.cc:377] Logged in from keytab as kudu/127.5.61.3@KRBTEST.COM (short username kudu)
I20260809 20:17:06.511914 5817 webserver.cc:460] Webserver started at http://127.5.61.3:43685/ using document root <none> and password file <none>
I20260809 20:17:06.512581 5817 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:06.512683 5817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:06.512943 5817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:06.515903 5817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "01942906017d46248e85540553d53eea"
format_stamp: "Formatted at 2026-08-09 20:17:06 on dist-test-slave-6k22"
server_key: "3953edaf3568e580438a46ad4081ee2d"
server_key_iv: "fd906ff7111096150cefe23b890b7a4b"
server_key_version: "encryptionkey@0"
I20260809 20:17:06.516683 5817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "01942906017d46248e85540553d53eea"
format_stamp: "Formatted at 2026-08-09 20:17:06 on dist-test-slave-6k22"
server_key: "3953edaf3568e580438a46ad4081ee2d"
server_key_iv: "fd906ff7111096150cefe23b890b7a4b"
server_key_version: "encryptionkey@0"
I20260809 20:17:06.521849 5817 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 20:17:06.525276 5839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:06.526544 5817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 20:17:06.526710 5817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "01942906017d46248e85540553d53eea"
format_stamp: "Formatted at 2026-08-09 20:17:06 on dist-test-slave-6k22"
server_key: "3953edaf3568e580438a46ad4081ee2d"
server_key_iv: "fd906ff7111096150cefe23b890b7a4b"
server_key_version: "encryptionkey@0"
I20260809 20:17:06.526938 5817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:06.559026 5817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:06.562973 5817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:06.563270 5817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:06.564301 5817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:06.565662 5817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:06.565764 5817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:06.565924 5817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:06.565999 5817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:06.601928 5817 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:42127
I20260809 20:17:06.601979 5972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:42127 every 8 connection(s)
I20260809 20:17:06.603199 5817 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:17:06.604714 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 5817
I20260809 20:17:06.604919 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:17:06.605473 5364 external_mini_cluster.cc:1428] Setting key 1379c7851f42cfaa69a06c876aabc407
I20260809 20:17:06.613114 5652 heartbeater.cc:503] Master 127.5.61.62:36397 was elected leader, sending a full tablet report...
Aug 09 20:17:06 dist-test-slave-6k22 krb5kdc[5371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306626, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:06.624089 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:06.607196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:36309 (local address 127.5.61.62:36397)
0809 20:17:06.607644 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:06.607649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:06.607670 (+ 21us) server_negotiation.cc:407] Connection header received
0809 20:17:06.609949 (+ 2279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:06.609954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:06.610040 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:06.610153 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:06.611612 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:06.612357 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:06.614099 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:06.614690 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:06.614829 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:06.618098 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:06.618103 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:06.618106 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:06.618138 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:06.621122 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:06.621915 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:06.621919 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:06.621922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:06.622106 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:06.622517 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:06.622520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:06.622522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:06.622920 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:06.623043 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:06.623690 (+ 647us) server_negotiation.cc:299] Negotiation successful
0809 20:17:06.623850 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279}
I20260809 20:17:06.624965 5975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:06.606575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:36397 (local address 127.5.61.3:36309)
0809 20:17:06.607400 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:06.607453 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:06.609700 (+ 2247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:06.610392 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:06.610405 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:06.610904 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:06.611430 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:06.611446 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:06.612509 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:06.612514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:06.613951 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:06.613965 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:06.614954 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:06.614963 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:06.615158 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:06.615810 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:06.615836 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:06.617924 (+ 2088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:06.621277 (+ 3353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:06.621285 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:06.621298 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:06.621785 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:06.622226 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:06.622231 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:06.622236 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:06.622434 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:06.623036 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:06.623041 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:06.623320 (+ 279us) client_negotiation.cc:769] Sending connection context
0809 20:17:06.623546 (+ 226us) client_negotiation.cc:240] Negotiation successful
0809 20:17:06.623958 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":345,"threads_started":1}
I20260809 20:17:06.626241 5973 heartbeater.cc:347] Connected to a master server at 127.5.61.62:36397
I20260809 20:17:06.626472 5973 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:06.627086 5973 heartbeater.cc:511] Master 127.5.61.62:36397 requested a full tablet report, sending...
I20260809 20:17:06.628440 5427 ts_manager.cc:194] Registered new tserver with Master: 01942906017d46248e85540553d53eea (127.5.61.3:42127)
I20260809 20:17:06.629202 5427 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.3@KRBTEST.COM'} at 127.5.61.3:36309
I20260809 20:17:06.634205 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 20:17:06 dist-test-slave-6k22 krb5kdc[5371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306626, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 20:17:06 dist-test-slave-6k22 krb5kdc[5371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306626, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.61.1@KRBTEST.COM
I20260809 20:17:06.681281 5979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:06.665433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57742 (local address 127.5.61.1:33443)
0809 20:17:06.666154 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:06.666168 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:06.666552 (+ 384us) server_negotiation.cc:407] Connection header received
0809 20:17:06.666777 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:06.666785 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:06.667010 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:06.667177 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:06.668794 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:06.669817 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:06.671076 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:06.671722 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:06.671878 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:06.676043 (+ 4165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:06.676053 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:06.676066 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:06.676096 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:06.678686 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:06.679338 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:06.679344 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:06.679346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:06.679464 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:06.679893 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:06.679896 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:06.679898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:06.680215 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:06.680438 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:06.680764 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 20:17:06.680966 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":337,"threads_started":1}
W20260809 20:17:06.682682 5554 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57742
Aug 09 20:17:06 dist-test-slave-6k22 krb5kdc[5371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 20:17:06 dist-test-slave-6k22 krb5kdc[5371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.1@KRBTEST.COM
I20260809 20:17:06.727414 5979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:06.714101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57756 (local address 127.5.61.1:33443)
0809 20:17:06.714343 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:06.714347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:06.714470 (+ 123us) server_negotiation.cc:407] Connection header received
0809 20:17:06.714616 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:06.714620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:06.714713 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:06.714851 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:06.716471 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:06.717152 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:06.718459 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:06.718862 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:06.718953 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:06.722635 (+ 3682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:06.722640 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:06.722643 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:06.722670 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:06.725322 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:06.725872 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:06.725876 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:06.725878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:06.725986 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:06.726283 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:06.726286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:06.726288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:06.726500 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:06.726669 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:06.726917 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 20:17:06.727068 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 20:17:06.729298 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 5502
I20260809 20:17:06.741415 5979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:06.730316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57772 (local address 127.5.61.1:33443)
0809 20:17:06.730547 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:06.730553 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:06.730751 (+ 198us) server_negotiation.cc:407] Connection header received
0809 20:17:06.730879 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:06.730882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:06.730937 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:06.731027 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:06.732270 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:06.733069 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:06.734193 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:06.734588 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:06.734679 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:06.736806 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:06.736810 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:06.736813 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:06.736840 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:06.739189 (+ 2349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:06.739828 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:06.739832 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:06.739834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:06.739967 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:06.740272 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:06.740275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:06.740277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:06.740503 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:06.740673 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:06.740896 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 20:17:06.741035 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 20:17:06.742249 5646 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:06.829087 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 5502
I20260809 20:17:06.848589 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 5662
Aug 09 20:17:06 dist-test-slave-6k22 krb5kdc[5371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.2@KRBTEST.COM
I20260809 20:17:06.864192 5987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:06.849444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41512 (local address 127.5.61.2:44045)
0809 20:17:06.849962 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:06.849969 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:06.849994 (+ 25us) server_negotiation.cc:407] Connection header received
0809 20:17:06.850071 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:06.850077 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:06.850284 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:06.850460 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:06.851666 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:06.852656 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:06.853901 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:06.854453 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:06.854591 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:06.858613 (+ 4022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:06.858622 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:06.858635 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:06.858667 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:06.861315 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:06.861945 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:06.861952 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:06.861954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:06.862080 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:06.862477 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:06.862482 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:06.862485 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:06.862826 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:06.863153 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:06.863473 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 20:17:06.863905 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":275,"threads_started":1}
I20260809 20:17:06.865515 5803 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:06.948939 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 5662
I20260809 20:17:06.970144 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 5817
Aug 09 20:17:06 dist-test-slave-6k22 krb5kdc[5371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.3@KRBTEST.COM
I20260809 20:17:06.986397 5989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:06.971265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49502 (local address 127.5.61.3:42127)
0809 20:17:06.971998 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:06.972005 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:06.972033 (+ 28us) server_negotiation.cc:407] Connection header received
0809 20:17:06.972098 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:06.972105 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:06.972311 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:06.972454 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:06.973662 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:06.974689 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:06.976077 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:06.976624 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:06.976723 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:06.980720 (+ 3997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:06.980731 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:06.980745 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:06.980777 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:06.983682 (+ 2905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:06.984243 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:06.984249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:06.984251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:06.984382 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:06.984703 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:06.984706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:06.984708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:06.985033 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:06.985359 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:06.985713 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 20:17:06.985996 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":402,"threads_started":1}
I20260809 20:17:06.987748 5968 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:07.071986 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 5817
I20260809 20:17:07.094182 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 5386
I20260809 20:17:07.095432 5463 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:07.197011 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 5386
2026-08-09T20:17:07Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4115 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:17:07 dist-test-slave-6k22 krb5kdc[5994](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:17:07 dist-test-slave-6k22 krb5kdc[5994](info): set up 2 sockets
Aug 09 20:17:07 dist-test-slave-6k22 krb5kdc[5994](info): commencing operation
krb5kdc: starting...
W20260809 20:17:08.359601 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:17:08 dist-test-slave-6k22 krb5kdc[5994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:17:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:17:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:17:08.543206 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:41725
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:41459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:41725
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:08.852442 6009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:08.852741 6009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:08.852910 6009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:08.860073 6009 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:17:08.860179 6009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:08.860260 6009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:08.860371 6009 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:17:08.860433 6009 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:17:08.870014 6009 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:41459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:41725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:41725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:08.871282 6009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:08.872613 6009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:08.884266 6014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:08.884280 6017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:08.884323 6015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:08.885445 6009 server_base.cc:1034] running on GCE node
I20260809 20:17:08.886243 6009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:08.887594 6009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:08.888827 6009 hybrid_clock.cc:648] HybridClock initialized: now 1786306628888777 us; error 36 us; skew 500 ppm
I20260809 20:17:08.889164 6009 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:08 dist-test-slave-6k22 krb5kdc[5994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306628, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:08.898025 6009 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:17:08.899549 6009 webserver.cc:460] Webserver started at http://127.5.61.62:35525/ using document root <none> and password file <none>
I20260809 20:17:08.900292 6009 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:08.900432 6009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:08.900751 6009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:08.903689 6009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "6040140e8aa34d94acfd362a852953af"
format_stamp: "Formatted at 2026-08-09 20:17:08 on dist-test-slave-6k22"
server_key: "9972458b3106ad9e4768a016fe7b106b"
server_key_iv: "0637ff66c4969ce45d030c7df06bdf0a"
server_key_version: "encryptionkey@0"
I20260809 20:17:08.904445 6009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "6040140e8aa34d94acfd362a852953af"
format_stamp: "Formatted at 2026-08-09 20:17:08 on dist-test-slave-6k22"
server_key: "9972458b3106ad9e4768a016fe7b106b"
server_key_iv: "0637ff66c4969ce45d030c7df06bdf0a"
server_key_version: "encryptionkey@0"
I20260809 20:17:08.909441 6009 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 20:17:08.912700 6026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:08.913911 6009 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 20:17:08.914094 6009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "6040140e8aa34d94acfd362a852953af"
format_stamp: "Formatted at 2026-08-09 20:17:08 on dist-test-slave-6k22"
server_key: "9972458b3106ad9e4768a016fe7b106b"
server_key_iv: "0637ff66c4969ce45d030c7df06bdf0a"
server_key_version: "encryptionkey@0"
I20260809 20:17:08.914292 6009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:08.933512 6009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:08.939221 6009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:08.939591 6009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:08.963478 6009 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:41725
I20260809 20:17:08.963516 6085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:41725 every 8 connection(s)
I20260809 20:17:08.965309 6009 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:17:08.969066 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 6009
I20260809 20:17:08.969302 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:17:08.970160 6088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:08.970243 5364 external_mini_cluster.cc:1428] Setting key b3586fa11b2c87b46d428a3cd4513a41
I20260809 20:17:08.986476 6088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6040140e8aa34d94acfd362a852953af: Bootstrap starting.
Aug 09 20:17:08 dist-test-slave-6k22 krb5kdc[5994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:08.990114 6088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6040140e8aa34d94acfd362a852953af: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:08.991528 6088 log.cc:824] T 00000000000000000000000000000000 P 6040140e8aa34d94acfd362a852953af: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:08.994853 6088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6040140e8aa34d94acfd362a852953af: No bootstrap required, opened a new log
I20260809 20:17:08.997556 6091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:08.972653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41106 (local address 127.5.61.62:41725)
0809 20:17:08.973564 (+ 911us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:08.973583 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:08.973632 (+ 49us) server_negotiation.cc:407] Connection header received
0809 20:17:08.975620 (+ 1988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:08.975648 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:08.976142 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:08.976556 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:08.978949 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:08.980894 (+ 1945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:08.982840 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:08.983998 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:08.984166 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:08.989137 (+ 4971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:08.989148 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:08.989169 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:08.989217 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:08.993153 (+ 3936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:08.993807 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:08.993815 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:08.993822 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:08.994031 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:08.994383 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:08.994387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:08.994390 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:08.995085 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:08.995381 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:08.995795 (+ 414us) server_negotiation.cc:299] Negotiation successful
0809 20:17:08.996219 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":381,"threads_started":1}
I20260809 20:17:09.003631 6088 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6040140e8aa34d94acfd362a852953af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6040140e8aa34d94acfd362a852953af" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41725 } }
I20260809 20:17:09.003968 6088 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6040140e8aa34d94acfd362a852953af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:09.004040 6088 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6040140e8aa34d94acfd362a852953af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6040140e8aa34d94acfd362a852953af, State: Initialized, Role: FOLLOWER
I20260809 20:17:09.004583 6088 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6040140e8aa34d94acfd362a852953af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6040140e8aa34d94acfd362a852953af" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41725 } }
I20260809 20:17:09.004804 6088 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6040140e8aa34d94acfd362a852953af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:17:09.004904 6088 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6040140e8aa34d94acfd362a852953af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:17:09.005015 6088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6040140e8aa34d94acfd362a852953af [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:09.007387 6088 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6040140e8aa34d94acfd362a852953af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6040140e8aa34d94acfd362a852953af" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41725 } }
I20260809 20:17:09.007884 6088 leader_election.cc:302] T 00000000000000000000000000000000 P 6040140e8aa34d94acfd362a852953af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6040140e8aa34d94acfd362a852953af; no voters:
I20260809 20:17:09.008419 6088 leader_election.cc:288] T 00000000000000000000000000000000 P 6040140e8aa34d94acfd362a852953af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:17:09.008553 6095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6040140e8aa34d94acfd362a852953af [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:09.009253 6095 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6040140e8aa34d94acfd362a852953af [term 1 LEADER]: Becoming Leader. State: Replica: 6040140e8aa34d94acfd362a852953af, State: Running, Role: LEADER
I20260809 20:17:09.009817 6095 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6040140e8aa34d94acfd362a852953af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6040140e8aa34d94acfd362a852953af" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41725 } }
I20260809 20:17:09.010576 6088 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6040140e8aa34d94acfd362a852953af [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:17:09.012889 6096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6040140e8aa34d94acfd362a852953af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6040140e8aa34d94acfd362a852953af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6040140e8aa34d94acfd362a852953af" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41725 } } }
I20260809 20:17:09.012849 6098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6040140e8aa34d94acfd362a852953af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6040140e8aa34d94acfd362a852953af. Latest consensus state: current_term: 1 leader_uuid: "6040140e8aa34d94acfd362a852953af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6040140e8aa34d94acfd362a852953af" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41725 } } }
I20260809 20:17:09.013355 6096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6040140e8aa34d94acfd362a852953af [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:09.013368 6098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6040140e8aa34d94acfd362a852953af [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:09.015635 6103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:17:09.021225 6103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:17:09.029376 6103 catalog_manager.cc:1347] Generated new cluster ID: 81a97ef1a22e4429a4de7ac2607cdbfe
I20260809 20:17:09.029491 6103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:17:09.045550 6103 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:17:09.046805 6103 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:17:09.064399 6103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6040140e8aa34d94acfd362a852953af: Generated new TSK 0
I20260809 20:17:09.065286 6103 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.61.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
I20260809 20:17:09.124119 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:41725
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:41459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:09.447297 6118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:09.447631 6118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:09.447804 6118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:09.454676 6118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:09.454823 6118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:09.455008 6118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:17:09.465071 6118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:41459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:41725
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:09.466572 6118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:09.467989 6118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:09.480921 6125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:09.480949 6128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:09.481016 6126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:09.482136 6118 server_base.cc:1034] running on GCE node
I20260809 20:17:09.482707 6118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:09.483656 6118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:09.484881 6118 hybrid_clock.cc:648] HybridClock initialized: now 1786306629484840 us; error 47 us; skew 500 ppm
I20260809 20:17:09.485196 6118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:09 dist-test-slave-6k22 krb5kdc[5994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306629, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:09.488355 6118 init.cc:377] Logged in from keytab as kudu/127.5.61.1@KRBTEST.COM (short username kudu)
I20260809 20:17:09.489851 6118 webserver.cc:460] Webserver started at http://127.5.61.1:36811/ using document root <none> and password file <none>
I20260809 20:17:09.490510 6118 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:09.490651 6118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:09.490985 6118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:09.494004 6118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "488bc227187e496cbe488be5c92dec00"
format_stamp: "Formatted at 2026-08-09 20:17:09 on dist-test-slave-6k22"
server_key: "0cff956fb84f45c1f3ae41cb23cb6ebb"
server_key_iv: "ca6a1e144bfdf3858365dfac0da6e455"
server_key_version: "encryptionkey@0"
I20260809 20:17:09.494778 6118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "488bc227187e496cbe488be5c92dec00"
format_stamp: "Formatted at 2026-08-09 20:17:09 on dist-test-slave-6k22"
server_key: "0cff956fb84f45c1f3ae41cb23cb6ebb"
server_key_iv: "ca6a1e144bfdf3858365dfac0da6e455"
server_key_version: "encryptionkey@0"
I20260809 20:17:09.499719 6118 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260809 20:17:09.502985 6135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:09.504194 6118 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 20:17:09.504367 6118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "488bc227187e496cbe488be5c92dec00"
format_stamp: "Formatted at 2026-08-09 20:17:09 on dist-test-slave-6k22"
server_key: "0cff956fb84f45c1f3ae41cb23cb6ebb"
server_key_iv: "ca6a1e144bfdf3858365dfac0da6e455"
server_key_version: "encryptionkey@0"
I20260809 20:17:09.504565 6118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:09.518332 6118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:09.522261 6118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:09.522562 6118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:09.523469 6118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:09.524709 6118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:09.524814 6118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:09.524957 6118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:09.525035 6118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:09.558012 6118 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:40905
I20260809 20:17:09.558058 6264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:40905 every 8 connection(s)
I20260809 20:17:09.559299 6118 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:17:09.562626 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 6118
I20260809 20:17:09.562963 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:17:09.563815 5364 external_mini_cluster.cc:1428] Setting key 26d5bf4592656febd9846be109e14491
Aug 09 20:17:09 dist-test-slave-6k22 krb5kdc[5994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306629, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:09.579859 6267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:09.561935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:47971 (local address 127.5.61.62:41725)
0809 20:17:09.562411 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:09.562415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:09.563611 (+ 1196us) server_negotiation.cc:407] Connection header received
0809 20:17:09.565640 (+ 2029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:09.565646 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:09.565750 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:09.565876 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:09.567354 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:09.568154 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:09.569734 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:09.570181 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:09.570320 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:09.574104 (+ 3784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:09.574109 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:09.574112 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:09.574143 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:09.576770 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:09.577544 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:09.577551 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:09.577559 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:09.577673 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:09.578114 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:09.578120 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:09.578123 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:09.578423 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:09.578595 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:09.579251 (+ 656us) server_negotiation.cc:299] Negotiation successful
0809 20:17:09.579469 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":269,"threads_started":1}
WARNING: no policy specified for kudu/127.5.61.2@KRBTEST.COM; defaulting to no policy
I20260809 20:17:09.580786 6268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:09.562476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41725 (local address 127.5.61.1:47971)
0809 20:17:09.563429 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:09.563484 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:09.565376 (+ 1892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:09.566098 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:09.566106 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:09.566632 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:09.567167 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:09.567184 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:09.568328 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:09.568331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:09.569581 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:09.569592 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:09.570415 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:09.570424 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:09.570649 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:09.571328 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:09.571349 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:09.573892 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:09.576919 (+ 3027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:09.576929 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:09.576946 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:09.577415 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:09.577815 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:09.577820 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:09.577824 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:09.578004 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:09.578554 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:09.578563 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:09.578892 (+ 329us) client_negotiation.cc:769] Sending connection context
0809 20:17:09.579149 (+ 257us) client_negotiation.cc:240] Negotiation successful
0809 20:17:09.579690 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":376,"threads_started":1}
I20260809 20:17:09.582357 6265 heartbeater.cc:347] Connected to a master server at 127.5.61.62:41725
I20260809 20:17:09.582666 6265 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:09.583422 6265 heartbeater.cc:511] Master 127.5.61.62:41725 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.2@KRBTEST.COM" created.
I20260809 20:17:09.585687 6048 ts_manager.cc:194] Registered new tserver with Master: 488bc227187e496cbe488be5c92dec00 (127.5.61.1:40905)
I20260809 20:17:09.587078 6048 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.1@KRBTEST.COM'} at 127.5.61.1:47971
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
I20260809 20:17:09.605899 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:41725
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:41459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:09.928879 6272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:09.929185 6272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:09.929307 6272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:09.936609 6272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:09.936720 6272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:09.936837 6272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:17:09.946456 6272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:41459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:41725
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:09.947768 6272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:09.949115 6272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:09.961387 6277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:09.961422 6281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:09.962616 6283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:09.963686 6272 server_base.cc:1034] running on GCE node
I20260809 20:17:09.964409 6272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:09.965590 6272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:09.966827 6272 hybrid_clock.cc:648] HybridClock initialized: now 1786306629966755 us; error 62 us; skew 500 ppm
I20260809 20:17:09.967181 6272 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:09 dist-test-slave-6k22 krb5kdc[5994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306629, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:09.975325 6272 init.cc:377] Logged in from keytab as kudu/127.5.61.2@KRBTEST.COM (short username kudu)
I20260809 20:17:09.980721 6272 webserver.cc:460] Webserver started at http://127.5.61.2:45287/ using document root <none> and password file <none>
I20260809 20:17:09.981493 6272 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:09.981647 6272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:09.981966 6272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:09.984962 6272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "5d768476cf9f481c93a026c96a41e821"
format_stamp: "Formatted at 2026-08-09 20:17:09 on dist-test-slave-6k22"
server_key: "187c5280264af8d9624411fd56de83ed"
server_key_iv: "e0adbc50a45ee77d13a93ca929fead4b"
server_key_version: "encryptionkey@0"
I20260809 20:17:09.985765 6272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "5d768476cf9f481c93a026c96a41e821"
format_stamp: "Formatted at 2026-08-09 20:17:09 on dist-test-slave-6k22"
server_key: "187c5280264af8d9624411fd56de83ed"
server_key_iv: "e0adbc50a45ee77d13a93ca929fead4b"
server_key_version: "encryptionkey@0"
I20260809 20:17:09.990948 6272 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 20:17:09.994400 6290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:09.995759 6272 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:17:09.995924 6272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "5d768476cf9f481c93a026c96a41e821"
format_stamp: "Formatted at 2026-08-09 20:17:09 on dist-test-slave-6k22"
server_key: "187c5280264af8d9624411fd56de83ed"
server_key_iv: "e0adbc50a45ee77d13a93ca929fead4b"
server_key_version: "encryptionkey@0"
I20260809 20:17:09.996121 6272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:10.016536 6272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:10.020804 6272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:10.021095 6272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:10.022056 6272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:10.023449 6272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:10.023557 6272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:10.023751 6272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:10.023828 6272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:10.057704 6272 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:46731
I20260809 20:17:10.057747 6425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:46731 every 8 connection(s)
I20260809 20:17:10.059000 6272 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:17:10.062433 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 6272
I20260809 20:17:10.062696 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:17:10.063274 5364 external_mini_cluster.cc:1428] Setting key 325678aa0c60d2f3486e3bd77cf4a9c7
Aug 09 20:17:10 dist-test-slave-6k22 krb5kdc[5994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306629, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:10.079941 6267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:10.061642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:51019 (local address 127.5.61.62:41725)
0809 20:17:10.061828 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:10.061833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:10.063223 (+ 1390us) server_negotiation.cc:407] Connection header received
0809 20:17:10.065699 (+ 2476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:10.065704 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:10.065800 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:10.065908 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:10.067799 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.068504 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.070082 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.070558 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.070691 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:10.074162 (+ 3471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:10.074167 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:10.074170 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:10.074200 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:10.076880 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:10.077597 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:10.077604 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:10.077612 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:10.077745 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:10.078189 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:10.078201 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:10.078203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:10.078501 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:10.078666 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:10.079315 (+ 649us) server_negotiation.cc:299] Negotiation successful
0809 20:17:10.079545 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
WARNING: no policy specified for kudu/127.5.61.3@KRBTEST.COM; defaulting to no policy
I20260809 20:17:10.080960 6428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:10.062164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41725 (local address 127.5.61.2:51019)
0809 20:17:10.063066 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:10.063111 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:10.065407 (+ 2296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:10.066159 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:10.066167 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:10.066974 (+ 807us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:10.067550 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:10.067633 (+ 83us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.068671 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.068677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:10.069935 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:10.069947 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.070799 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.070805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:10.071025 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:10.071725 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:10.071753 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:10.073934 (+ 2181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:10.077022 (+ 3088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:10.077031 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:10.077046 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:10.077469 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:10.077859 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:10.077864 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:10.077867 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:10.078057 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:10.078630 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:10.078637 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:10.078956 (+ 319us) client_negotiation.cc:769] Sending connection context
0809 20:17:10.079231 (+ 275us) client_negotiation.cc:240] Negotiation successful
0809 20:17:10.079789 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":414,"threads_started":1}
I20260809 20:17:10.082476 6426 heartbeater.cc:347] Connected to a master server at 127.5.61.62:41725
I20260809 20:17:10.082852 6426 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.3@KRBTEST.COM" created.
I20260809 20:17:10.083969 6426 heartbeater.cc:511] Master 127.5.61.62:41725 requested a full tablet report, sending...
I20260809 20:17:10.085696 6048 ts_manager.cc:194] Registered new tserver with Master: 5d768476cf9f481c93a026c96a41e821 (127.5.61.2:46731)
I20260809 20:17:10.086570 6048 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.2@KRBTEST.COM'} at 127.5.61.2:51019
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
I20260809 20:17:10.106160 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:41725
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:41459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:10.434237 6431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:10.434525 6431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:10.434677 6431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:10.441864 6431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:10.441957 6431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:10.442147 6431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:17:10.451289 6431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:41459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:41725
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:10.452533 6431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:10.453805 6431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:10.466638 6440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:10.466673 6438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:10.466754 6437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:10.467978 6431 server_base.cc:1034] running on GCE node
I20260809 20:17:10.468762 6431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:10.470077 6431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:10.471304 6431 hybrid_clock.cc:648] HybridClock initialized: now 1786306630471233 us; error 60 us; skew 500 ppm
I20260809 20:17:10.471694 6431 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:10 dist-test-slave-6k22 krb5kdc[5994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306630, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:10.474685 6431 init.cc:377] Logged in from keytab as kudu/127.5.61.3@KRBTEST.COM (short username kudu)
I20260809 20:17:10.476209 6431 webserver.cc:460] Webserver started at http://127.5.61.3:41807/ using document root <none> and password file <none>
I20260809 20:17:10.477048 6431 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:10.477196 6431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:10.477502 6431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:10.480455 6431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "da49a7d0df7f42c69f7d9ab544f704f8"
format_stamp: "Formatted at 2026-08-09 20:17:10 on dist-test-slave-6k22"
server_key: "187e4eecb4fb50957d54b285fa32f415"
server_key_iv: "1c3f23d047de30ed1a8283299c69b45e"
server_key_version: "encryptionkey@0"
I20260809 20:17:10.481232 6431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "da49a7d0df7f42c69f7d9ab544f704f8"
format_stamp: "Formatted at 2026-08-09 20:17:10 on dist-test-slave-6k22"
server_key: "187e4eecb4fb50957d54b285fa32f415"
server_key_iv: "1c3f23d047de30ed1a8283299c69b45e"
server_key_version: "encryptionkey@0"
I20260809 20:17:10.486542 6431 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 20:17:10.490167 6448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:10.491679 6431 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:17:10.491853 6431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "da49a7d0df7f42c69f7d9ab544f704f8"
format_stamp: "Formatted at 2026-08-09 20:17:10 on dist-test-slave-6k22"
server_key: "187e4eecb4fb50957d54b285fa32f415"
server_key_iv: "1c3f23d047de30ed1a8283299c69b45e"
server_key_version: "encryptionkey@0"
I20260809 20:17:10.492079 6431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:10.510497 6431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:10.514744 6431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:10.515033 6431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:10.516085 6431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:10.517393 6431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:10.517493 6431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:10.517638 6431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:10.517707 6431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:10.553395 6431 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:41745
I20260809 20:17:10.553453 6580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:41745 every 8 connection(s)
I20260809 20:17:10.554689 6431 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:17:10.563052 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 6431
I20260809 20:17:10.563265 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:17:10.563772 5364 external_mini_cluster.cc:1428] Setting key 325464c69ed17abf577e98afd018de3f
Aug 09 20:17:10 dist-test-slave-6k22 krb5kdc[5994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306630, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:10.574594 6267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:10.557335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:50903 (local address 127.5.61.62:41725)
0809 20:17:10.557574 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:10.557598 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:10.559055 (+ 1457us) server_negotiation.cc:407] Connection header received
0809 20:17:10.560970 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:10.560976 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:10.561057 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:10.561194 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:10.562718 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.563691 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.565434 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.565996 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.566131 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:10.569599 (+ 3468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:10.569604 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:10.569607 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:10.569638 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:10.572074 (+ 2436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:10.572739 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:10.572743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:10.572745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:10.572859 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:10.573225 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:10.573228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:10.573230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:10.573476 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:10.573585 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:10.574184 (+ 599us) server_negotiation.cc:299] Negotiation successful
0809 20:17:10.574339 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 20:17:10.575516 6583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:10.558015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41725 (local address 127.5.61.3:50903)
0809 20:17:10.558873 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:10.558918 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:10.560703 (+ 1785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:10.561476 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:10.561485 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:10.561988 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:10.562526 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:10.562543 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.563856 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.563880 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:10.565252 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:10.565264 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.566200 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.566205 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:10.566405 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:10.567162 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:10.567185 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:10.569394 (+ 2209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:10.572217 (+ 2823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:10.572223 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:10.572234 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:10.572641 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:10.572960 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:10.572963 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:10.572966 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:10.573136 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:10.573580 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:10.573586 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:10.573869 (+ 283us) client_negotiation.cc:769] Sending connection context
0809 20:17:10.574088 (+ 219us) client_negotiation.cc:240] Negotiation successful
0809 20:17:10.574451 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":397,"threads_started":1}
I20260809 20:17:10.576907 6581 heartbeater.cc:347] Connected to a master server at 127.5.61.62:41725
I20260809 20:17:10.577242 6581 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:10.578130 6581 heartbeater.cc:511] Master 127.5.61.62:41725 requested a full tablet report, sending...
I20260809 20:17:10.579692 6048 ts_manager.cc:194] Registered new tserver with Master: da49a7d0df7f42c69f7d9ab544f704f8 (127.5.61.3:41745)
I20260809 20:17:10.580488 6048 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.3@KRBTEST.COM'} at 127.5.61.3:50903
I20260809 20:17:10.589833 6265 heartbeater.cc:503] Master 127.5.61.62:41725 was elected leader, sending a full tablet report...
I20260809 20:17:10.591441 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 20:17:10.610929 6267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:10.598330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41108 (local address 127.5.61.62:41725)
0809 20:17:10.598497 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:10.598501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:10.599043 (+ 542us) server_negotiation.cc:407] Connection header received
0809 20:17:10.599239 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:10.599245 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:10.599337 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:10.599436 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:10.601023 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.601718 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.602972 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.603403 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.603540 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:10.605820 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:10.605826 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:10.605838 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:10.605874 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:10.608450 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:10.609122 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:10.609127 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:10.609131 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:10.609254 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:10.609655 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:10.609659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:10.609660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:10.609962 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:10.610139 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:10.610425 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 20:17:10.610575 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 20:17:10.619069 6048 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:17:10.627035 6048 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:17:10.650887 6595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:10.641633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:46731 (local address 127.0.0.1:55448)
0809 20:17:10.642428 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:10.642478 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:10.642740 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:10.643769 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:10.643776 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:10.643815 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:10.644305 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:10.644319 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.646485 (+ 2166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.646490 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:10.648342 (+ 1852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:10.648380 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.649781 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.649789 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:10.650010 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:10.650075 (+ 65us) client_negotiation.cc:769] Sending connection context
0809 20:17:10.650176 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 20:17:10.650386 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"mutex_wait_us":23,"thread_start_us":386,"threads_started":1}
I20260809 20:17:10.651592 6596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:10.641695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55448 (local address 127.5.61.2:46731)
0809 20:17:10.643038 (+ 1343us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:10.643049 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:10.643080 (+ 31us) server_negotiation.cc:407] Connection header received
0809 20:17:10.643190 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:10.643198 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:10.643499 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:10.644541 (+ 1042us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:10.644801 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.646309 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.648540 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.649505 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.650746 (+ 1241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:10.650979 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:10.651057 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 20:17:10.651247 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1169,"thread_start_us":1052,"threads_started":1}
I20260809 20:17:10.652931 6599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:10.643643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:40905 (local address 127.0.0.1:37534)
0809 20:17:10.644420 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:10.644436 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:10.644530 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:10.645671 (+ 1141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:10.645675 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:10.645699 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:10.646117 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:10.646129 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.648167 (+ 2038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.648173 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:10.650543 (+ 2370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:10.650558 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.652261 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.652268 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:10.652451 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:10.652484 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 20:17:10.652583 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 20:17:10.652668 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":49,"spinlock_wait_cycles":3328,"thread_start_us":570,"threads_started":1}
I20260809 20:17:10.653643 6600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:10.644060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37534 (local address 127.5.61.1:40905)
0809 20:17:10.645133 (+ 1073us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:10.645145 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:10.645174 (+ 29us) server_negotiation.cc:407] Connection header received
0809 20:17:10.645268 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:10.645276 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:10.645483 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:10.645663 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:10.646365 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.648023 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.650705 (+ 2682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.652039 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.652181 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:10.652410 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:10.653258 (+ 848us) server_negotiation.cc:299] Negotiation successful
0809 20:17:10.653382 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":915,"thread_start_us":736,"threads_started":1}
I20260809 20:17:10.654269 6597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:10.642170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:41745 (local address 127.0.0.1:35910)
0809 20:17:10.643279 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:10.643295 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:10.643420 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:10.646108 (+ 2688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:10.646112 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:10.646135 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:10.646572 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:10.646588 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.648641 (+ 2053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.648646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:10.650796 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:10.650808 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.653280 (+ 2472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.653287 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:10.653488 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:10.653526 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 20:17:10.653625 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 20:17:10.653782 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":996,"mutex_wait_us":34,"thread_start_us":822,"threads_started":1}
I20260809 20:17:10.655501 6598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:10.642577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35910 (local address 127.5.61.3:41745)
0809 20:17:10.645512 (+ 2935us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:10.645518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:10.645542 (+ 24us) server_negotiation.cc:407] Connection header received
0809 20:17:10.645646 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:10.645652 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:10.645875 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:10.646792 (+ 917us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:10.647054 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.648429 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.650988 (+ 2559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.653021 (+ 2033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.654679 (+ 1658us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:10.654894 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:10.654960 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 20:17:10.655113 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2793,"thread_start_us":2628,"threads_started":1}
I20260809 20:17:10.656272 6346 tablet_service.cc:1467] Processing CreateTablet for tablet 7adb10e0f4d24051a011791e5442e547 (DEFAULT_TABLE table=test-table [id=ea18707cac8747dca23c775b3fbf6344]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:10.656852 6190 tablet_service.cc:1467] Processing CreateTablet for tablet 7adb10e0f4d24051a011791e5442e547 (DEFAULT_TABLE table=test-table [id=ea18707cac8747dca23c775b3fbf6344]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:10.657826 6346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7adb10e0f4d24051a011791e5442e547. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:10.658237 6190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7adb10e0f4d24051a011791e5442e547. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:10.658967 6506 tablet_service.cc:1467] Processing CreateTablet for tablet 7adb10e0f4d24051a011791e5442e547 (DEFAULT_TABLE table=test-table [id=ea18707cac8747dca23c775b3fbf6344]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:10.660475 6506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7adb10e0f4d24051a011791e5442e547. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:10.669868 6602 tablet_bootstrap.cc:492] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821: Bootstrap starting.
I20260809 20:17:10.673434 6603 tablet_bootstrap.cc:492] T 7adb10e0f4d24051a011791e5442e547 P 488bc227187e496cbe488be5c92dec00: Bootstrap starting.
I20260809 20:17:10.674005 6602 tablet_bootstrap.cc:654] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:10.675340 6602 log.cc:824] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:10.676240 6604 tablet_bootstrap.cc:492] T 7adb10e0f4d24051a011791e5442e547 P da49a7d0df7f42c69f7d9ab544f704f8: Bootstrap starting.
I20260809 20:17:10.677544 6603 tablet_bootstrap.cc:654] T 7adb10e0f4d24051a011791e5442e547 P 488bc227187e496cbe488be5c92dec00: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:10.679109 6604 tablet_bootstrap.cc:654] T 7adb10e0f4d24051a011791e5442e547 P da49a7d0df7f42c69f7d9ab544f704f8: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:10.679100 6603 log.cc:824] T 7adb10e0f4d24051a011791e5442e547 P 488bc227187e496cbe488be5c92dec00: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:10.680339 6604 log.cc:824] T 7adb10e0f4d24051a011791e5442e547 P da49a7d0df7f42c69f7d9ab544f704f8: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:10.682493 6602 tablet_bootstrap.cc:492] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821: No bootstrap required, opened a new log
I20260809 20:17:10.682780 6602 ts_tablet_manager.cc:1397] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.000s
I20260809 20:17:10.682809 6603 tablet_bootstrap.cc:492] T 7adb10e0f4d24051a011791e5442e547 P 488bc227187e496cbe488be5c92dec00: No bootstrap required, opened a new log
I20260809 20:17:10.683035 6603 ts_tablet_manager.cc:1397] T 7adb10e0f4d24051a011791e5442e547 P 488bc227187e496cbe488be5c92dec00: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260809 20:17:10.683391 6604 tablet_bootstrap.cc:492] T 7adb10e0f4d24051a011791e5442e547 P da49a7d0df7f42c69f7d9ab544f704f8: No bootstrap required, opened a new log
I20260809 20:17:10.683681 6604 ts_tablet_manager.cc:1397] T 7adb10e0f4d24051a011791e5442e547 P da49a7d0df7f42c69f7d9ab544f704f8: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.002s
I20260809 20:17:10.689924 6603 raft_consensus.cc:348] T 7adb10e0f4d24051a011791e5442e547 P 488bc227187e496cbe488be5c92dec00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d768476cf9f481c93a026c96a41e821" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 46731 } } peers { permanent_uuid: "da49a7d0df7f42c69f7d9ab544f704f8" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41745 } } peers { permanent_uuid: "488bc227187e496cbe488be5c92dec00" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40905 } }
I20260809 20:17:10.690302 6603 raft_consensus.cc:374] T 7adb10e0f4d24051a011791e5442e547 P 488bc227187e496cbe488be5c92dec00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:10.690428 6603 raft_consensus.cc:729] T 7adb10e0f4d24051a011791e5442e547 P 488bc227187e496cbe488be5c92dec00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 488bc227187e496cbe488be5c92dec00, State: Initialized, Role: FOLLOWER
I20260809 20:17:10.691068 6603 consensus_queue.cc:260] T 7adb10e0f4d24051a011791e5442e547 P 488bc227187e496cbe488be5c92dec00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d768476cf9f481c93a026c96a41e821" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 46731 } } peers { permanent_uuid: "da49a7d0df7f42c69f7d9ab544f704f8" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41745 } } peers { permanent_uuid: "488bc227187e496cbe488be5c92dec00" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40905 } }
I20260809 20:17:10.693547 6602 raft_consensus.cc:348] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d768476cf9f481c93a026c96a41e821" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 46731 } } peers { permanent_uuid: "da49a7d0df7f42c69f7d9ab544f704f8" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41745 } } peers { permanent_uuid: "488bc227187e496cbe488be5c92dec00" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40905 } }
I20260809 20:17:10.694041 6602 raft_consensus.cc:374] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:10.694188 6602 raft_consensus.cc:729] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d768476cf9f481c93a026c96a41e821, State: Initialized, Role: FOLLOWER
I20260809 20:17:10.694566 6603 ts_tablet_manager.cc:1428] T 7adb10e0f4d24051a011791e5442e547 P 488bc227187e496cbe488be5c92dec00: Time spent starting tablet: real 0.011s user 0.004s sys 0.008s
I20260809 20:17:10.694839 6602 consensus_queue.cc:260] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d768476cf9f481c93a026c96a41e821" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 46731 } } peers { permanent_uuid: "da49a7d0df7f42c69f7d9ab544f704f8" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41745 } } peers { permanent_uuid: "488bc227187e496cbe488be5c92dec00" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40905 } }
I20260809 20:17:10.695899 6604 raft_consensus.cc:348] T 7adb10e0f4d24051a011791e5442e547 P da49a7d0df7f42c69f7d9ab544f704f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d768476cf9f481c93a026c96a41e821" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 46731 } } peers { permanent_uuid: "da49a7d0df7f42c69f7d9ab544f704f8" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41745 } } peers { permanent_uuid: "488bc227187e496cbe488be5c92dec00" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40905 } }
I20260809 20:17:10.696343 6604 raft_consensus.cc:374] T 7adb10e0f4d24051a011791e5442e547 P da49a7d0df7f42c69f7d9ab544f704f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:10.696404 6426 heartbeater.cc:503] Master 127.5.61.62:41725 was elected leader, sending a full tablet report...
I20260809 20:17:10.696472 6604 raft_consensus.cc:729] T 7adb10e0f4d24051a011791e5442e547 P da49a7d0df7f42c69f7d9ab544f704f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da49a7d0df7f42c69f7d9ab544f704f8, State: Initialized, Role: FOLLOWER
I20260809 20:17:10.697032 6604 consensus_queue.cc:260] T 7adb10e0f4d24051a011791e5442e547 P da49a7d0df7f42c69f7d9ab544f704f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d768476cf9f481c93a026c96a41e821" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 46731 } } peers { permanent_uuid: "da49a7d0df7f42c69f7d9ab544f704f8" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41745 } } peers { permanent_uuid: "488bc227187e496cbe488be5c92dec00" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40905 } }
I20260809 20:17:10.698076 6581 heartbeater.cc:503] Master 127.5.61.62:41725 was elected leader, sending a full tablet report...
I20260809 20:17:10.699735 6602 ts_tablet_manager.cc:1428] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821: Time spent starting tablet: real 0.017s user 0.015s sys 0.001s
I20260809 20:17:10.699885 6604 ts_tablet_manager.cc:1428] T 7adb10e0f4d24051a011791e5442e547 P da49a7d0df7f42c69f7d9ab544f704f8: Time spent starting tablet: real 0.016s user 0.012s sys 0.001s
I20260809 20:17:10.718909 6610 raft_consensus.cc:482] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:17:10.719204 6610 raft_consensus.cc:504] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d768476cf9f481c93a026c96a41e821" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 46731 } } peers { permanent_uuid: "da49a7d0df7f42c69f7d9ab544f704f8" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41745 } } peers { permanent_uuid: "488bc227187e496cbe488be5c92dec00" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40905 } }
I20260809 20:17:10.721431 6610 leader_election.cc:288] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da49a7d0df7f42c69f7d9ab544f704f8 (127.5.61.3:41745), 488bc227187e496cbe488be5c92dec00 (127.5.61.1:40905)
I20260809 20:17:10.728659 6598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:10.721767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:56973 (local address 127.5.61.3:41745)
0809 20:17:10.721975 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:10.721982 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:10.722568 (+ 586us) server_negotiation.cc:407] Connection header received
0809 20:17:10.722781 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:10.722788 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:10.722875 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:10.722967 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:10.723778 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.724734 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.726675 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.727608 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.727702 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:10.727785 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:10.728257 (+ 472us) server_negotiation.cc:299] Negotiation successful
0809 20:17:10.728403 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 20:17:10.728760 6612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:10.721795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:41745 (local address 127.5.61.2:56973)
0809 20:17:10.722462 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:10.722478 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:10.722641 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:10.723086 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:10.723095 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:10.723118 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:10.723626 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:10.723642 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.724883 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.724889 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:10.726518 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:10.726530 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.727832 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.727839 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:10.728027 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:10.728121 (+ 94us) client_negotiation.cc:769] Sending connection context
0809 20:17:10.728200 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 20:17:10.728290 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"mutex_wait_us":28,"spinlock_wait_cycles":60672,"thread_start_us":374,"threads_started":1}
I20260809 20:17:10.728783 6600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:10.722832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:44985 (local address 127.5.61.1:40905)
0809 20:17:10.723035 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:10.723042 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:10.723289 (+ 247us) server_negotiation.cc:407] Connection header received
0809 20:17:10.723455 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:10.723463 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:10.723545 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:10.723670 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:10.724306 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.725124 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.726913 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.727830 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.727968 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:10.728067 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:10.728431 (+ 364us) server_negotiation.cc:299] Negotiation successful
0809 20:17:10.728557 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 20:17:10.729188 6613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:10.722582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:40905 (local address 127.5.61.2:44985)
0809 20:17:10.723179 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:10.723192 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:10.723326 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:10.723716 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:10.723720 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:10.723748 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:10.724148 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:10.724159 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.725282 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.725285 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:10.726753 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:10.726767 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.728039 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.728044 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:10.728227 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:10.728267 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 20:17:10.728357 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 20:17:10.728489 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":2,"thread_start_us":299,"threads_started":1}
I20260809 20:17:10.729913 6531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adb10e0f4d24051a011791e5442e547" candidate_uuid: "5d768476cf9f481c93a026c96a41e821" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da49a7d0df7f42c69f7d9ab544f704f8" is_pre_election: true
I20260809 20:17:10.730134 6213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adb10e0f4d24051a011791e5442e547" candidate_uuid: "5d768476cf9f481c93a026c96a41e821" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "488bc227187e496cbe488be5c92dec00" is_pre_election: true
I20260809 20:17:10.730670 6531 raft_consensus.cc:2454] T 7adb10e0f4d24051a011791e5442e547 P da49a7d0df7f42c69f7d9ab544f704f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d768476cf9f481c93a026c96a41e821 in term 0.
I20260809 20:17:10.730669 6213 raft_consensus.cc:2454] T 7adb10e0f4d24051a011791e5442e547 P 488bc227187e496cbe488be5c92dec00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d768476cf9f481c93a026c96a41e821 in term 0.
I20260809 20:17:10.731546 6293 leader_election.cc:302] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d768476cf9f481c93a026c96a41e821, da49a7d0df7f42c69f7d9ab544f704f8; no voters:
I20260809 20:17:10.732016 6610 raft_consensus.cc:2781] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:17:10.732172 6610 raft_consensus.cc:482] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:17:10.732278 6610 raft_consensus.cc:3037] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:10.734567 6610 raft_consensus.cc:504] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d768476cf9f481c93a026c96a41e821" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 46731 } } peers { permanent_uuid: "da49a7d0df7f42c69f7d9ab544f704f8" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41745 } } peers { permanent_uuid: "488bc227187e496cbe488be5c92dec00" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40905 } }
I20260809 20:17:10.735162 6610 leader_election.cc:288] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821 [CANDIDATE]: Term 1 election: Requested vote from peers da49a7d0df7f42c69f7d9ab544f704f8 (127.5.61.3:41745), 488bc227187e496cbe488be5c92dec00 (127.5.61.1:40905)
I20260809 20:17:10.735628 6531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adb10e0f4d24051a011791e5442e547" candidate_uuid: "5d768476cf9f481c93a026c96a41e821" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da49a7d0df7f42c69f7d9ab544f704f8"
I20260809 20:17:10.735764 6213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adb10e0f4d24051a011791e5442e547" candidate_uuid: "5d768476cf9f481c93a026c96a41e821" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "488bc227187e496cbe488be5c92dec00"
I20260809 20:17:10.735841 6531 raft_consensus.cc:3037] T 7adb10e0f4d24051a011791e5442e547 P da49a7d0df7f42c69f7d9ab544f704f8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:10.735987 6213 raft_consensus.cc:3037] T 7adb10e0f4d24051a011791e5442e547 P 488bc227187e496cbe488be5c92dec00 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:10.738054 6531 raft_consensus.cc:2454] T 7adb10e0f4d24051a011791e5442e547 P da49a7d0df7f42c69f7d9ab544f704f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d768476cf9f481c93a026c96a41e821 in term 1.
I20260809 20:17:10.738234 6213 raft_consensus.cc:2454] T 7adb10e0f4d24051a011791e5442e547 P 488bc227187e496cbe488be5c92dec00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d768476cf9f481c93a026c96a41e821 in term 1.
I20260809 20:17:10.738477 6293 leader_election.cc:302] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d768476cf9f481c93a026c96a41e821, da49a7d0df7f42c69f7d9ab544f704f8; no voters:
I20260809 20:17:10.738761 6610 raft_consensus.cc:2781] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:10.739255 6610 raft_consensus.cc:686] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821 [term 1 LEADER]: Becoming Leader. State: Replica: 5d768476cf9f481c93a026c96a41e821, State: Running, Role: LEADER
I20260809 20:17:10.739877 6610 consensus_queue.cc:237] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d768476cf9f481c93a026c96a41e821" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 46731 } } peers { permanent_uuid: "da49a7d0df7f42c69f7d9ab544f704f8" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41745 } } peers { permanent_uuid: "488bc227187e496cbe488be5c92dec00" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40905 } }
I20260809 20:17:10.744971 6047 catalog_manager.cc:5495] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d768476cf9f481c93a026c96a41e821 (127.5.61.2). New cstate: current_term: 1 leader_uuid: "5d768476cf9f481c93a026c96a41e821" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d768476cf9f481c93a026c96a41e821" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 46731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da49a7d0df7f42c69f7d9ab544f704f8" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "488bc227187e496cbe488be5c92dec00" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40905 } health_report { overall_health: UNKNOWN } } }
Aug 09 20:17:10 dist-test-slave-6k22 krb5kdc[5994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.1@KRBTEST.COM
I20260809 20:17:10.781551 6600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:10.766441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37550 (local address 127.5.61.1:40905)
0809 20:17:10.766643 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:10.766648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:10.767191 (+ 543us) server_negotiation.cc:407] Connection header received
0809 20:17:10.767395 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:10.767400 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:10.767496 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:10.767628 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:10.769198 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.769928 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.771204 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.771753 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.771895 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:10.775885 (+ 3990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:10.775897 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:10.775912 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:10.775942 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:10.778711 (+ 2769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:10.779392 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:10.779400 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:10.779402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:10.779530 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:10.779959 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:10.779965 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:10.779969 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:10.780331 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:10.780654 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:10.780956 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 20:17:10.781185 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260809 20:17:10.806320 6582 tablet.cc:2369] T 7adb10e0f4d24051a011791e5442e547 P da49a7d0df7f42c69f7d9ab544f704f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 20:17:10 dist-test-slave-6k22 krb5kdc[5994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306630, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260809 20:17:10.810981 6427 tablet.cc:2369] T 7adb10e0f4d24051a011791e5442e547 P 5d768476cf9f481c93a026c96a41e821: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 20:17:10.811678 6266 tablet.cc:2369] T 7adb10e0f4d24051a011791e5442e547 P 488bc227187e496cbe488be5c92dec00: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 20:17:10 dist-test-slave-6k22 krb5kdc[5994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306630, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.61.1@KRBTEST.COM
I20260809 20:17:10.830034 6600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:10.816592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37556 (local address 127.5.61.1:40905)
0809 20:17:10.816796 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:10.816805 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:10.817210 (+ 405us) server_negotiation.cc:407] Connection header received
0809 20:17:10.817448 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:10.817452 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:10.817514 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:10.817620 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:10.819009 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.819836 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.821067 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.821465 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.821596 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:10.825425 (+ 3829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:10.825435 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:10.825438 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:10.825470 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:10.827948 (+ 2478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:10.828492 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:10.828495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:10.828498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:10.828615 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:10.828953 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:10.828956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:10.828958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:10.829214 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:10.829412 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:10.829620 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 20:17:10.829775 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 20:17:10 dist-test-slave-6k22 krb5kdc[5994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 20:17:10 dist-test-slave-6k22 krb5kdc[5994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.1@KRBTEST.COM
I20260809 20:17:10.875679 6600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:10.862231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37568 (local address 127.5.61.1:40905)
0809 20:17:10.862464 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:10.862471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:10.862637 (+ 166us) server_negotiation.cc:407] Connection header received
0809 20:17:10.862863 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:10.862873 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:10.862937 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:10.863044 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:10.864503 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.865194 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.866441 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.866859 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.866993 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:10.870820 (+ 3827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:10.870824 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:10.870828 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:10.870859 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:10.873531 (+ 2672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:10.874083 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:10.874087 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:10.874089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:10.874210 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:10.874502 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:10.874518 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:10.874520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:10.874808 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:10.875000 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:10.875187 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 20:17:10.875359 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260809 20:17:10.877880 6167 tablet_service.cc:2908] Rejecting scan request for tablet 7adb10e0f4d24051a011791e5442e547: Uninitialized: clean time has not yet been initialized
I20260809 20:17:10.881747 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 6118
I20260809 20:17:10.893572 6600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:10.882663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37574 (local address 127.5.61.1:40905)
0809 20:17:10.882875 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:10.882880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:10.883100 (+ 220us) server_negotiation.cc:407] Connection header received
0809 20:17:10.883217 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:10.883220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:10.883286 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:10.883382 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:10.884523 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.885170 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.886266 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:10.886627 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:10.886717 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:10.888746 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:10.888750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:10.888753 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:10.888779 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:10.891051 (+ 2272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:10.891682 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:10.891688 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:10.891693 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:10.891817 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:10.892183 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:10.892189 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:10.892197 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:10.892462 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:10.892758 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:10.893053 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 20:17:10.893264 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 20:17:10.894146 6259 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:10.983009 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 6118
I20260809 20:17:11.003371 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 6272
Aug 09 20:17:11 dist-test-slave-6k22 krb5kdc[5994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.2@KRBTEST.COM
I20260809 20:17:11.018416 6596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:11.004253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55462 (local address 127.5.61.2:46731)
0809 20:17:11.004457 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:11.004462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:11.004480 (+ 18us) server_negotiation.cc:407] Connection header received
0809 20:17:11.004620 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:11.004624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:11.004697 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:11.004788 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:11.005951 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:11.006698 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:11.008027 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:11.008402 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:11.008493 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:11.012518 (+ 4025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:11.012530 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:11.012545 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:11.012577 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:11.015944 (+ 3367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:11.016585 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:11.016592 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:11.016594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:11.016731 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:11.017050 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:11.017053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:11.017055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:11.017448 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:11.017664 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:11.017873 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 20:17:11.018037 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 20:17:11.019106 6419 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:11.109889 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 6272
I20260809 20:17:11.132107 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 6431
Aug 09 20:17:11 dist-test-slave-6k22 krb5kdc[5994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.3@KRBTEST.COM
I20260809 20:17:11.146688 6598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:11.133014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35912 (local address 127.5.61.3:41745)
0809 20:17:11.133200 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:11.133208 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:11.133309 (+ 101us) server_negotiation.cc:407] Connection header received
0809 20:17:11.133542 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:11.133547 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:11.133620 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:11.133725 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:11.134980 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:11.135794 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:11.137086 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:11.137504 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:11.137646 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:11.141750 (+ 4104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:11.141763 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:11.141777 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:11.141810 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:11.144489 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:11.145061 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:11.145068 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:11.145070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:11.145213 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:11.145540 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:11.145543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:11.145545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:11.145864 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:11.146066 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:11.146301 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 20:17:11.146456 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 20:17:11.147349 6576 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:11.235872 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 6431
I20260809 20:17:11.257565 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 6009
I20260809 20:17:11.258813 6080 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:11.367312 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 6009
2026-08-09T20:17:11Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4170 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:17:11 dist-test-slave-6k22 krb5kdc[6640](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:17:11 dist-test-slave-6k22 krb5kdc[6640](info): set up 2 sockets
Aug 09 20:17:11 dist-test-slave-6k22 krb5kdc[6640](info): commencing operation
krb5kdc: starting...
W20260809 20:17:12.535189 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:17:12 dist-test-slave-6k22 krb5kdc[6640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:17:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:17:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:17:12.719687 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:44159
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:40781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:44159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:13.039373 6654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:13.039661 6654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:13.039747 6654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:13.046442 6654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:17:13.046535 6654 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 20:17:13.046574 6654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:13.046635 6654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:13.046684 6654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:17:13.046736 6654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:17:13.056011 6654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:40781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:44159
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:44159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:13.057143 6654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:13.058398 6654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:13.070051 6662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:13.070010 6664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:13.070088 6661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:13.070969 6654 server_base.cc:1034] running on GCE node
I20260809 20:17:13.071733 6654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:13.073051 6654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:13.074288 6654 hybrid_clock.cc:648] HybridClock initialized: now 1786306633074228 us; error 38 us; skew 500 ppm
I20260809 20:17:13.074608 6654 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:13 dist-test-slave-6k22 krb5kdc[6640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306633, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:13.077644 6654 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:17:13.079003 6654 webserver.cc:460] Webserver started at http://127.5.61.62:38529/ using document root <none> and password file <none>
I20260809 20:17:13.079735 6654 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:13.079871 6654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:13.080200 6654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:13.083026 6654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "972bc05eeaa441e48fb9d9f10e92b4a2"
format_stamp: "Formatted at 2026-08-09 20:17:13 on dist-test-slave-6k22"
server_key: "9a61bf7f57d47ab78da767435a943bb5"
server_key_iv: "b5052b64b2329e1ba28e578df6eb1e8e"
server_key_version: "encryptionkey@0"
I20260809 20:17:13.083885 6654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "972bc05eeaa441e48fb9d9f10e92b4a2"
format_stamp: "Formatted at 2026-08-09 20:17:13 on dist-test-slave-6k22"
server_key: "9a61bf7f57d47ab78da767435a943bb5"
server_key_iv: "b5052b64b2329e1ba28e578df6eb1e8e"
server_key_version: "encryptionkey@0"
I20260809 20:17:13.088636 6654 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 20:17:13.091969 6673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:13.093200 6654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 20:17:13.093367 6654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "972bc05eeaa441e48fb9d9f10e92b4a2"
format_stamp: "Formatted at 2026-08-09 20:17:13 on dist-test-slave-6k22"
server_key: "9a61bf7f57d47ab78da767435a943bb5"
server_key_iv: "b5052b64b2329e1ba28e578df6eb1e8e"
server_key_version: "encryptionkey@0"
I20260809 20:17:13.093573 6654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:13.112820 6654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:13.116529 6654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:13.116782 6654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:13.137972 6654 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:44159
I20260809 20:17:13.137992 6740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:44159 every 8 connection(s)
I20260809 20:17:13.139251 6654 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:17:13.142733 6741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:13.145553 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 6654
I20260809 20:17:13.145794 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:17:13.146646 5364 external_mini_cluster.cc:1428] Setting key b04b95557dfe509da78d4d6970be119f
I20260809 20:17:13.155373 6741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2: Bootstrap starting.
I20260809 20:17:13.159106 6741 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:13.160444 6741 log.cc:824] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2: Log is configured to *not* fsync() on all Append() calls
Aug 09 20:17:13 dist-test-slave-6k22 krb5kdc[6640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:13.163388 6741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2: No bootstrap required, opened a new log
I20260809 20:17:13.169701 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:13.148805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35684 (local address 127.5.61.62:44159)
0809 20:17:13.149555 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:13.149566 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:13.149604 (+ 38us) server_negotiation.cc:407] Connection header received
0809 20:17:13.151315 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:13.151346 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:13.151777 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:13.152090 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:13.153848 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:13.155061 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:13.156781 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:13.157445 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:13.157565 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:13.162627 (+ 5062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:13.162634 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:13.162646 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:13.162681 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:13.165808 (+ 3127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:13.166586 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:13.166592 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:13.166597 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:13.166753 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:13.167138 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:13.167141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:13.167143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:13.167680 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:13.167901 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:13.168348 (+ 447us) server_negotiation.cc:299] Negotiation successful
0809 20:17:13.168707 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":354,"threads_started":1}
I20260809 20:17:13.172685 6741 raft_consensus.cc:348] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "972bc05eeaa441e48fb9d9f10e92b4a2" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44159 } }
I20260809 20:17:13.173028 6741 raft_consensus.cc:374] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:13.173127 6741 raft_consensus.cc:729] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972bc05eeaa441e48fb9d9f10e92b4a2, State: Initialized, Role: FOLLOWER
I20260809 20:17:13.173659 6741 consensus_queue.cc:260] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "972bc05eeaa441e48fb9d9f10e92b4a2" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44159 } }
I20260809 20:17:13.173895 6741 raft_consensus.cc:388] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:17:13.174039 6741 raft_consensus.cc:482] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:17:13.174198 6741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:13.176558 6741 raft_consensus.cc:504] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "972bc05eeaa441e48fb9d9f10e92b4a2" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44159 } }
I20260809 20:17:13.177039 6741 leader_election.cc:302] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 972bc05eeaa441e48fb9d9f10e92b4a2; no voters:
I20260809 20:17:13.177609 6741 leader_election.cc:288] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:17:13.177693 6747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:13.178440 6747 raft_consensus.cc:686] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2 [term 1 LEADER]: Becoming Leader. State: Replica: 972bc05eeaa441e48fb9d9f10e92b4a2, State: Running, Role: LEADER
I20260809 20:17:13.179050 6747 consensus_queue.cc:237] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "972bc05eeaa441e48fb9d9f10e92b4a2" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44159 } }
I20260809 20:17:13.179649 6741 sys_catalog.cc:564] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:17:13.181762 6749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 972bc05eeaa441e48fb9d9f10e92b4a2. Latest consensus state: current_term: 1 leader_uuid: "972bc05eeaa441e48fb9d9f10e92b4a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "972bc05eeaa441e48fb9d9f10e92b4a2" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44159 } } }
I20260809 20:17:13.182117 6749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2 [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:13.183455 6752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:17:13.183713 6748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "972bc05eeaa441e48fb9d9f10e92b4a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "972bc05eeaa441e48fb9d9f10e92b4a2" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44159 } } }
I20260809 20:17:13.184034 6748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2 [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:13.188728 6752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:17:13.197517 6752 catalog_manager.cc:1347] Generated new cluster ID: 666e586dea5447e0b4ffe7b46c38bebf
I20260809 20:17:13.197618 6752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:17:13.217672 6752 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:17:13.218977 6752 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:17:13.229997 6752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2: Generated new TSK 0
I20260809 20:17:13.230779 6752 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.61.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
I20260809 20:17:13.277560 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:44159
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:40781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:13.601567 6771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:13.601831 6771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:13.602005 6771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:13.609156 6771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:13.609282 6771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:13.609438 6771 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 20:17:13.609529 6771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:17:13.618937 6771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:40781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.61.62:44159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:13.620301 6771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:13.621603 6771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:13.634685 6781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:13.634692 6783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:13.634758 6780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:13.635367 6771 server_base.cc:1034] running on GCE node
I20260809 20:17:13.636031 6771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:13.636893 6771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:13.638135 6771 hybrid_clock.cc:648] HybridClock initialized: now 1786306633638065 us; error 45 us; skew 500 ppm
I20260809 20:17:13.638440 6771 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:13 dist-test-slave-6k22 krb5kdc[6640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306633, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:13.641538 6771 init.cc:377] Logged in from keytab as kudu/127.5.61.1@KRBTEST.COM (short username kudu)
I20260809 20:17:13.642903 6771 webserver.cc:460] Webserver started at http://127.5.61.1:34459/ using document root <none> and password file <none>
I20260809 20:17:13.643632 6771 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:13.643771 6771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:13.644106 6771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:13.647003 6771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "7ee8075f0e5d4e69909921c388bfeea9"
format_stamp: "Formatted at 2026-08-09 20:17:13 on dist-test-slave-6k22"
server_key: "cda4c892de24ec08e507ee7f47f22c7e"
server_key_iv: "b73110d7c82299c2aacf553e837924b5"
server_key_version: "encryptionkey@0"
I20260809 20:17:13.647833 6771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "7ee8075f0e5d4e69909921c388bfeea9"
format_stamp: "Formatted at 2026-08-09 20:17:13 on dist-test-slave-6k22"
server_key: "cda4c892de24ec08e507ee7f47f22c7e"
server_key_iv: "b73110d7c82299c2aacf553e837924b5"
server_key_version: "encryptionkey@0"
I20260809 20:17:13.653313 6771 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 20:17:13.656971 6794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:13.658313 6771 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:17:13.658521 6771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "7ee8075f0e5d4e69909921c388bfeea9"
format_stamp: "Formatted at 2026-08-09 20:17:13 on dist-test-slave-6k22"
server_key: "cda4c892de24ec08e507ee7f47f22c7e"
server_key_iv: "b73110d7c82299c2aacf553e837924b5"
server_key_version: "encryptionkey@0"
I20260809 20:17:13.658774 6771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:13.679955 6771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:13.684340 6771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:13.684746 6771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:13.687870 6771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:13.687975 6771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:13.688186 6771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:13.688279 6771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 20:17:13 dist-test-slave-6k22 krb5kdc[6640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306633, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:13.712893 6806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:13.688492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:42999 (local address 127.5.61.62:44159)
0809 20:17:13.696015 (+ 7523us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:13.696020 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:13.696037 (+ 17us) server_negotiation.cc:407] Connection header received
0809 20:17:13.696089 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:13.696092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:13.696148 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:13.696270 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:13.697854 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:13.698782 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:13.700949 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:13.701918 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:13.702101 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:13.706002 (+ 3901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:13.706010 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:13.706014 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:13.706056 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:13.709486 (+ 3430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:13.710550 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:13.710558 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:13.710563 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:13.710711 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:13.711192 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:13.711196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:13.711200 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:13.711505 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:13.711692 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:13.712325 (+ 633us) server_negotiation.cc:299] Negotiation successful
0809 20:17:13.712546 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7400,"thread_start_us":7385,"threads_started":1}
I20260809 20:17:13.713816 6807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:13.689078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:44159 (local address 127.5.61.1:42999)
0809 20:17:13.689908 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:13.689952 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:13.692050 (+ 2098us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:13.696520 (+ 4470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:13.696528 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:13.697085 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:13.697678 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:13.697693 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:13.698949 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:13.698953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:13.700757 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:13.700774 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:13.702158 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:13.702163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:13.702356 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:13.703046 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:13.703068 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:13.705771 (+ 2703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:13.709622 (+ 3851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:13.709629 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:13.709651 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:13.710328 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:13.710844 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:13.710850 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:13.710859 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:13.711057 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:13.711665 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:13.711672 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:13.711986 (+ 314us) client_negotiation.cc:769] Sending connection context
0809 20:17:13.712237 (+ 251us) client_negotiation.cc:240] Negotiation successful
0809 20:17:13.712717 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":360,"threads_started":1}
I20260809 20:17:13.738770 6853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:13.719777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:44159 (local address 127.5.61.1:60463)
0809 20:17:13.724089 (+ 4312us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:13.724106 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:13.724218 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:13.724613 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:13.724616 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:13.725026 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:13.725295 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:13.725303 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:13.727058 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:13.727065 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:13.728702 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:13.728714 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:13.729817 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:13.729826 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:13.730029 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:13.730724 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:13.730741 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:13.731433 (+ 692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:13.735173 (+ 3740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:13.735182 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:13.735189 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:13.736015 (+ 826us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:13.736568 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:13.736578 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:13.736583 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:13.736774 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:13.737534 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:13.737538 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:13.737769 (+ 231us) client_negotiation.cc:769] Sending connection context
0809 20:17:13.738127 (+ 358us) client_negotiation.cc:240] Negotiation successful
0809 20:17:13.738447 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4160,"thread_start_us":4024,"threads_started":1}
I20260809 20:17:13.739183 6806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:13.719849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:60463 (local address 127.5.61.62:44159)
0809 20:17:13.720068 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:13.720076 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:13.724204 (+ 4128us) server_negotiation.cc:407] Connection header received
0809 20:17:13.724344 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:13.724352 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:13.724464 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:13.724605 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:13.725433 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:13.726890 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:13.728894 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:13.729552 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:13.729744 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:13.731648 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:13.731659 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:13.731663 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:13.731692 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:13.734991 (+ 3299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:13.736225 (+ 1234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:13.736235 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:13.736241 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:13.736424 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:13.736915 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:13.736920 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:13.736927 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:13.737329 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:13.737482 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:13.738303 (+ 821us) server_negotiation.cc:299] Negotiation successful
0809 20:17:13.738647 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 20:17:13.750684 6771 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:37181
I20260809 20:17:13.750715 6936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:37181 every 8 connection(s)
I20260809 20:17:13.752022 6771 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:17:13.754150 6937 heartbeater.cc:347] Connected to a master server at 127.5.61.62:44159
I20260809 20:17:13.754375 6937 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:13.755015 6937 heartbeater.cc:511] Master 127.5.61.62:44159 requested a full tablet report, sending...
I20260809 20:17:13.756945 6692 ts_manager.cc:194] Registered new tserver with Master: 7ee8075f0e5d4e69909921c388bfeea9 (127.5.61.1:37181)
I20260809 20:17:13.757091 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 6771
I20260809 20:17:13.757328 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:17:13.757957 5364 external_mini_cluster.cc:1428] Setting key e78ee2b8f40ec622cf2dc4556dd80654
I20260809 20:17:13.758514 6692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.1@KRBTEST.COM'} at 127.5.61.1:42999
WARNING: no policy specified for kudu/127.5.61.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
I20260809 20:17:13.798564 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:44159
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:40781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:14.133791 6942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:14.134060 6942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:14.134280 6942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:14.141680 6942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:14.141844 6942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:14.142011 6942 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 20:17:14.142176 6942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:17:14.152149 6942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:40781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.61.62:44159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:14.153368 6942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:14.154707 6942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:14.168440 6952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:14.168524 6948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:14.168496 6947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:14.169406 6942 server_base.cc:1034] running on GCE node
I20260809 20:17:14.170018 6942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:14.170816 6942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:14.172024 6942 hybrid_clock.cc:648] HybridClock initialized: now 1786306634171974 us; error 26 us; skew 500 ppm
I20260809 20:17:14.172305 6942 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:14 dist-test-slave-6k22 krb5kdc[6640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306634, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:14.176735 6942 init.cc:377] Logged in from keytab as kudu/127.5.61.2@KRBTEST.COM (short username kudu)
I20260809 20:17:14.179018 6942 webserver.cc:460] Webserver started at http://127.5.61.2:43333/ using document root <none> and password file <none>
I20260809 20:17:14.179924 6942 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:14.180053 6942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:14.180382 6942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:14.183256 6942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "38a9f7e8cb214a419b9d66349741d713"
format_stamp: "Formatted at 2026-08-09 20:17:14 on dist-test-slave-6k22"
server_key: "8a4bf3257ade075a67fb1f52d8d16248"
server_key_iv: "36be25e41782c353e403eb6967424898"
server_key_version: "encryptionkey@0"
I20260809 20:17:14.184087 6942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "38a9f7e8cb214a419b9d66349741d713"
format_stamp: "Formatted at 2026-08-09 20:17:14 on dist-test-slave-6k22"
server_key: "8a4bf3257ade075a67fb1f52d8d16248"
server_key_iv: "36be25e41782c353e403eb6967424898"
server_key_version: "encryptionkey@0"
I20260809 20:17:14.188980 6942 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 20:17:14.192341 6961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:14.193560 6942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:17:14.193734 6942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "38a9f7e8cb214a419b9d66349741d713"
format_stamp: "Formatted at 2026-08-09 20:17:14 on dist-test-slave-6k22"
server_key: "8a4bf3257ade075a67fb1f52d8d16248"
server_key_iv: "36be25e41782c353e403eb6967424898"
server_key_version: "encryptionkey@0"
I20260809 20:17:14.193969 6942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:14.217196 6942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:14.221079 6942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:14.221362 6942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:14.224073 6942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:14.224179 6942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:14.224319 6942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:14.224400 6942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 20:17:14 dist-test-slave-6k22 krb5kdc[6640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306634, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:14.248145 6806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:14.224725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:41999 (local address 127.5.61.62:44159)
0809 20:17:14.224960 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:14.224966 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:14.226375 (+ 1409us) server_negotiation.cc:407] Connection header received
0809 20:17:14.228625 (+ 2250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:14.228631 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:14.228719 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:14.228843 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:14.231195 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.232188 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.235362 (+ 3174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.235970 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.236106 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:14.241158 (+ 5052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:14.241163 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:14.241167 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:14.241199 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:14.243904 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:14.244977 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:14.244983 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:14.244989 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:14.245174 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:14.245861 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:14.245868 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:14.245874 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:14.246239 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:14.246392 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:14.247303 (+ 911us) server_negotiation.cc:299] Negotiation successful
0809 20:17:14.247650 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 20:17:14.248939 6973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:14.225418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:44159 (local address 127.5.61.2:41999)
0809 20:17:14.226217 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:14.226254 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:14.228350 (+ 2096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:14.229193 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:14.229207 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:14.230003 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:14.231019 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:14.231038 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.232390 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.232398 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:14.235199 (+ 2801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:14.235214 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.236449 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.236463 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:14.236813 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:14.237554 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:14.237593 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:14.240931 (+ 3338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:14.244038 (+ 3107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:14.244047 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:14.244064 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:14.244840 (+ 776us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:14.245331 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:14.245336 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:14.245341 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:14.245646 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:14.246390 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:14.246398 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:14.246763 (+ 365us) client_negotiation.cc:769] Sending connection context
0809 20:17:14.247145 (+ 382us) client_negotiation.cc:240] Negotiation successful
0809 20:17:14.247532 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":352,"threads_started":1}
I20260809 20:17:14.269174 7036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:14.254994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:44159 (local address 127.5.61.2:45173)
0809 20:17:14.255677 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:14.255696 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:14.255842 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:14.256138 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:14.256143 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:14.256602 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:14.256883 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:14.256892 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.257838 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.257845 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:14.258887 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:14.258899 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.259885 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.259892 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:14.260075 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:14.260760 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:14.260820 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:14.262243 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:14.266297 (+ 4054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:14.266302 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:14.266306 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:14.266791 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:14.267266 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:14.267269 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:14.267272 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:14.267389 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:14.268258 (+ 869us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:14.268265 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:14.268475 (+ 210us) client_negotiation.cc:769] Sending connection context
0809 20:17:14.268617 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 20:17:14.268835 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":364,"threads_started":1}
I20260809 20:17:14.269601 6806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:14.254903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:45173 (local address 127.5.61.62:44159)
0809 20:17:14.255046 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:14.255049 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:14.255778 (+ 729us) server_negotiation.cc:407] Connection header received
0809 20:17:14.255954 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:14.255964 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:14.256017 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:14.256112 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:14.257017 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.257690 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.259033 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.259697 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.259842 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:14.262460 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:14.262467 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:14.262474 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:14.262516 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:14.266129 (+ 3613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:14.266921 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:14.266929 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:14.266935 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:14.267138 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:14.267499 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:14.267507 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:14.267512 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:14.268100 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:14.268289 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:14.268783 (+ 494us) server_negotiation.cc:299] Negotiation successful
0809 20:17:14.269103 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 20:17:14.283479 6942 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:33415
I20260809 20:17:14.283521 7107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:33415 every 8 connection(s)
I20260809 20:17:14.284710 6942 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:17:14.285147 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 6942
I20260809 20:17:14.285362 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:17:14.285985 5364 external_mini_cluster.cc:1428] Setting key a061d90f50f42d704dd13578f2fb4862
I20260809 20:17:14.287166 7108 heartbeater.cc:347] Connected to a master server at 127.5.61.62:44159
I20260809 20:17:14.287410 7108 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:14.288194 7108 heartbeater.cc:511] Master 127.5.61.62:44159 requested a full tablet report, sending...
I20260809 20:17:14.289574 6691 ts_manager.cc:194] Registered new tserver with Master: 38a9f7e8cb214a419b9d66349741d713 (127.5.61.2:33415)
I20260809 20:17:14.290900 6691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.2@KRBTEST.COM'} at 127.5.61.2:41999
WARNING: no policy specified for kudu/127.5.61.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
I20260809 20:17:14.327093 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:44159
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:40781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:14.650419 7112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:14.650753 7112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:14.650925 7112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:14.658103 7112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:14.658246 7112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:14.658432 7112 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 20:17:14.658578 7112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:17:14.668159 7112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:40781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.61.62:44159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:14.669378 7112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:14.670717 7112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:14.684087 7117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:14.683977 7120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:14.684036 7118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:14.685511 7112 server_base.cc:1034] running on GCE node
I20260809 20:17:14.686196 7112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:14.687083 7112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:14.688315 7112 hybrid_clock.cc:648] HybridClock initialized: now 1786306634688240 us; error 55 us; skew 500 ppm
I20260809 20:17:14.688599 7112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:14 dist-test-slave-6k22 krb5kdc[6640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306634, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:14.691749 7112 init.cc:377] Logged in from keytab as kudu/127.5.61.3@KRBTEST.COM (short username kudu)
I20260809 20:17:14.693130 7112 webserver.cc:460] Webserver started at http://127.5.61.3:45303/ using document root <none> and password file <none>
I20260809 20:17:14.693859 7112 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:14.694012 7112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:14.694348 7112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:14.697408 7112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "7e98a98123714d3c97be8633bd03af03"
format_stamp: "Formatted at 2026-08-09 20:17:14 on dist-test-slave-6k22"
server_key: "816861be383924537eb936520deecdce"
server_key_iv: "bd459c3a9aa32ec1f6e05f2adcc85630"
server_key_version: "encryptionkey@0"
I20260809 20:17:14.698328 7112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "7e98a98123714d3c97be8633bd03af03"
format_stamp: "Formatted at 2026-08-09 20:17:14 on dist-test-slave-6k22"
server_key: "816861be383924537eb936520deecdce"
server_key_iv: "bd459c3a9aa32ec1f6e05f2adcc85630"
server_key_version: "encryptionkey@0"
I20260809 20:17:14.703508 7112 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 20:17:14.707100 7127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:14.708542 7112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:17:14.708777 7112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "7e98a98123714d3c97be8633bd03af03"
format_stamp: "Formatted at 2026-08-09 20:17:14 on dist-test-slave-6k22"
server_key: "816861be383924537eb936520deecdce"
server_key_iv: "bd459c3a9aa32ec1f6e05f2adcc85630"
server_key_version: "encryptionkey@0"
I20260809 20:17:14.709030 7112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:14.734221 7112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:14.738690 7112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:14.739009 7112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:14.742026 7112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:14.742146 7112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:14.742296 7112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:14.742400 7112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 20:17:14 dist-test-slave-6k22 krb5kdc[6640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306634, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:14.761313 6937 heartbeater.cc:503] Master 127.5.61.62:44159 was elected leader, sending a full tablet report...
I20260809 20:17:14.769294 6806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:14.742594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:58695 (local address 127.5.61.62:44159)
0809 20:17:14.742842 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:14.742852 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:14.744296 (+ 1444us) server_negotiation.cc:407] Connection header received
0809 20:17:14.746289 (+ 1993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:14.746295 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:14.746396 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:14.746520 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:14.748728 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.749809 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.754819 (+ 5010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.755475 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.755670 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:14.761373 (+ 5703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:14.761379 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:14.761383 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:14.761429 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:14.765044 (+ 3615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:14.766231 (+ 1187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:14.766235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:14.766237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:14.766380 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:14.766991 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:14.766994 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:14.766996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:14.767261 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:14.767381 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:14.768553 (+ 1172us) server_negotiation.cc:299] Negotiation successful
0809 20:17:14.768823 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260809 20:17:14.771885 7140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:14.743262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:44159 (local address 127.5.61.3:58695)
0809 20:17:14.744128 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:14.744166 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:14.746010 (+ 1844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:14.746780 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:14.746787 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:14.747549 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:14.748485 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:14.748514 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.750010 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.750019 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:14.754616 (+ 4597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:14.754638 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.755800 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.755813 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:14.756338 (+ 525us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:14.757301 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:14.757346 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:14.761096 (+ 3750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:14.765222 (+ 4126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:14.765235 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:14.765262 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:14.766073 (+ 811us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:14.766504 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:14.766512 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:14.766519 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:14.766849 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:14.767398 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:14.767410 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:14.767980 (+ 570us) client_negotiation.cc:769] Sending connection context
0809 20:17:14.768413 (+ 433us) client_negotiation.cc:240] Negotiation successful
0809 20:17:14.769144 (+ 731us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":423,"threads_started":1}
I20260809 20:17:14.795679 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:14.778898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:44159 (local address 127.5.61.3:50867)
0809 20:17:14.779845 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:14.779870 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:14.780025 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:14.780654 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:14.780657 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:14.781087 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:14.781373 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:14.781381 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.782978 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.782986 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:14.784907 (+ 1921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:14.784922 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.785936 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.785943 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:14.786103 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:14.786775 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:14.786847 (+ 72us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:14.788168 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:14.791886 (+ 3718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:14.791900 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:14.791906 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:14.792741 (+ 835us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:14.793224 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:14.793229 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:14.793233 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:14.793389 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:14.794243 (+ 854us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:14.794248 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:14.794513 (+ 265us) client_negotiation.cc:769] Sending connection context
0809 20:17:14.794698 (+ 185us) client_negotiation.cc:240] Negotiation successful
0809 20:17:14.795036 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"thread_start_us":600,"threads_started":1}
I20260809 20:17:14.796314 6806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:14.778840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:50867 (local address 127.5.61.62:44159)
0809 20:17:14.779028 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:14.779034 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:14.779973 (+ 939us) server_negotiation.cc:407] Connection header received
0809 20:17:14.780171 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:14.780177 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:14.780311 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:14.780464 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:14.781550 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.782789 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.785086 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.785740 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.785891 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:14.788352 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:14.788359 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:14.788364 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:14.788403 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:14.791716 (+ 3313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:14.792908 (+ 1192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:14.792915 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:14.792920 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:14.793091 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:14.793583 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:14.793593 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:14.793597 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:14.794067 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:14.794262 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:14.795789 (+ 1527us) server_negotiation.cc:299] Negotiation successful
0809 20:17:14.796043 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 20:17:14.802527 7112 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:37793
I20260809 20:17:14.802569 7275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:37793 every 8 connection(s)
I20260809 20:17:14.803853 7112 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:17:14.804493 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 7112
I20260809 20:17:14.804697 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:17:14.805132 5364 external_mini_cluster.cc:1428] Setting key ab424b9412130e7954931c7827c4e7e4
I20260809 20:17:14.806175 7277 heartbeater.cc:347] Connected to a master server at 127.5.61.62:44159
I20260809 20:17:14.806496 7277 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:14.807523 7277 heartbeater.cc:511] Master 127.5.61.62:44159 requested a full tablet report, sending...
I20260809 20:17:14.809161 6691 ts_manager.cc:194] Registered new tserver with Master: 7e98a98123714d3c97be8633bd03af03 (127.5.61.3:37793)
I20260809 20:17:14.809937 6691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.3@KRBTEST.COM'} at 127.5.61.3:58695
I20260809 20:17:14.820519 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 20:17:14 dist-test-slave-6k22 krb5kdc[6640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306634, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 20:17:14 dist-test-slave-6k22 krb5kdc[6640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306634, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:14.869596 6806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:14.854907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35700 (local address 127.5.61.62:44159)
0809 20:17:14.855131 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:14.855138 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:14.855309 (+ 171us) server_negotiation.cc:407] Connection header received
0809 20:17:14.855541 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:14.855546 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:14.855666 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:14.855775 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:14.857180 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.857917 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.859256 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.859764 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.859901 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:14.863936 (+ 4035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:14.863942 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:14.863945 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:14.863981 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:14.866376 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:14.867182 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:14.867190 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:14.867195 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:14.867347 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:14.867861 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:14.867874 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:14.867883 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:14.868206 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:14.868491 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:14.868892 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 20:17:14.869161 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 20:17:14.874184 6691 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:17:14.881793 6691 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:17:14.915642 7294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:14.894053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:33415 (local address 127.0.0.1:35960)
0809 20:17:14.895014 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:14.895057 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:14.895216 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:14.908017 (+ 12801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:14.908022 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:14.908049 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:14.908708 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:14.908716 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.910470 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.910477 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:14.912788 (+ 2311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:14.912800 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.914550 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.914559 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:14.914742 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:14.914794 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 20:17:14.914941 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 20:17:14.915249 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"mutex_wait_us":209,"thread_start_us":573,"threads_started":1}
I20260809 20:17:14.915844 7293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:14.894120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35960 (local address 127.5.61.2:33415)
0809 20:17:14.907379 (+ 13259us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:14.907385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:14.907405 (+ 20us) server_negotiation.cc:407] Connection header received
0809 20:17:14.907508 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:14.907514 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:14.907839 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:14.908038 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:14.908905 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.910322 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.913012 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.914329 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.914462 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:14.914660 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:14.915271 (+ 611us) server_negotiation.cc:299] Negotiation successful
0809 20:17:14.915455 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13101,"thread_start_us":12959,"threads_started":1}
I20260809 20:17:14.916668 7298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:14.895855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37534 (local address 127.5.61.1:37181)
0809 20:17:14.905891 (+ 10036us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:14.905901 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:14.906816 (+ 915us) server_negotiation.cc:407] Connection header received
0809 20:17:14.906949 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:14.906958 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:14.907323 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:14.907537 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:14.908927 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.910957 (+ 2030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.913683 (+ 2726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.915087 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.915206 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:14.915405 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:14.916150 (+ 745us) server_negotiation.cc:299] Negotiation successful
0809 20:17:14.916332 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9850,"thread_start_us":9709,"threads_started":1}
I20260809 20:17:14.917066 7296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:14.894547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:37793 (local address 127.0.0.1:53206)
0809 20:17:14.906219 (+ 11672us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:14.906241 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:14.906399 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:14.907625 (+ 1226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:14.907629 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:14.907653 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:14.908214 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:14.908227 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.910382 (+ 2155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.910388 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:14.913230 (+ 2842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:14.913245 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.915053 (+ 1808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.915060 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:14.916453 (+ 1393us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:14.916491 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 20:17:14.916568 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 20:17:14.916687 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11523,"mutex_wait_us":254,"thread_start_us":11711,"threads_started":1}
I20260809 20:17:14.917513 7295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:14.895377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37181 (local address 127.0.0.1:37534)
0809 20:17:14.906541 (+ 11164us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:14.906559 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:14.906678 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:14.907583 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:14.907592 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:14.907633 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:14.908701 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:14.908714 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.911118 (+ 2404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.911123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:14.913541 (+ 2418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:14.913552 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.915597 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.915606 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:14.915780 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:14.915813 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 20:17:14.915898 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 20:17:14.916918 (+ 1020us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11010,"mutex_wait_us":302,"thread_start_us":10546,"threads_started":1}
I20260809 20:17:14.917883 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:14.895359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53206 (local address 127.5.61.3:37793)
0809 20:17:14.907048 (+ 11689us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:14.907058 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:14.907086 (+ 28us) server_negotiation.cc:407] Connection header received
0809 20:17:14.907172 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:14.907179 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:14.907407 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:14.907984 (+ 577us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:14.908522 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.910200 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.913402 (+ 3202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:14.914836 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:14.914982 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:14.915218 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:14.917358 (+ 2140us) server_negotiation.cc:299] Negotiation successful
0809 20:17:14.917540 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11515,"thread_start_us":11324,"threads_started":1}
I20260809 20:17:14.919960 7018 tablet_service.cc:1467] Processing CreateTablet for tablet d475d6606b194f30bb79d271af762245 (DEFAULT_TABLE table=test-table [id=f87aee4c55cf41409e4a7fb3e3fdc946]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:14.920717 7189 tablet_service.cc:1467] Processing CreateTablet for tablet d475d6606b194f30bb79d271af762245 (DEFAULT_TABLE table=test-table [id=f87aee4c55cf41409e4a7fb3e3fdc946]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:14.921197 7018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d475d6606b194f30bb79d271af762245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:14.921806 7189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d475d6606b194f30bb79d271af762245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:14.926440 6859 tablet_service.cc:1467] Processing CreateTablet for tablet d475d6606b194f30bb79d271af762245 (DEFAULT_TABLE table=test-table [id=f87aee4c55cf41409e4a7fb3e3fdc946]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:14.927829 6859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d475d6606b194f30bb79d271af762245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:14.933583 7299 tablet_bootstrap.cc:492] T d475d6606b194f30bb79d271af762245 P 38a9f7e8cb214a419b9d66349741d713: Bootstrap starting.
I20260809 20:17:14.935945 7301 tablet_bootstrap.cc:492] T d475d6606b194f30bb79d271af762245 P 7e98a98123714d3c97be8633bd03af03: Bootstrap starting.
I20260809 20:17:14.937307 7299 tablet_bootstrap.cc:654] T d475d6606b194f30bb79d271af762245 P 38a9f7e8cb214a419b9d66349741d713: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:14.938697 7299 log.cc:824] T d475d6606b194f30bb79d271af762245 P 38a9f7e8cb214a419b9d66349741d713: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:14.939725 7301 tablet_bootstrap.cc:654] T d475d6606b194f30bb79d271af762245 P 7e98a98123714d3c97be8633bd03af03: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:14.941057 7304 tablet_bootstrap.cc:492] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9: Bootstrap starting.
I20260809 20:17:14.941247 7301 log.cc:824] T d475d6606b194f30bb79d271af762245 P 7e98a98123714d3c97be8633bd03af03: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:14.942665 7299 tablet_bootstrap.cc:492] T d475d6606b194f30bb79d271af762245 P 38a9f7e8cb214a419b9d66349741d713: No bootstrap required, opened a new log
I20260809 20:17:14.942955 7299 ts_tablet_manager.cc:1397] T d475d6606b194f30bb79d271af762245 P 38a9f7e8cb214a419b9d66349741d713: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 20:17:14.944376 7304 tablet_bootstrap.cc:654] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:14.944963 7301 tablet_bootstrap.cc:492] T d475d6606b194f30bb79d271af762245 P 7e98a98123714d3c97be8633bd03af03: No bootstrap required, opened a new log
I20260809 20:17:14.945218 7301 ts_tablet_manager.cc:1397] T d475d6606b194f30bb79d271af762245 P 7e98a98123714d3c97be8633bd03af03: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 20:17:14.945467 7304 log.cc:824] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:14.947817 7304 tablet_bootstrap.cc:492] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9: No bootstrap required, opened a new log
I20260809 20:17:14.948057 7304 ts_tablet_manager.cc:1397] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.005s
I20260809 20:17:14.953769 7299 raft_consensus.cc:348] T d475d6606b194f30bb79d271af762245 P 38a9f7e8cb214a419b9d66349741d713 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:14.954210 7299 raft_consensus.cc:374] T d475d6606b194f30bb79d271af762245 P 38a9f7e8cb214a419b9d66349741d713 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:14.954309 7299 raft_consensus.cc:729] T d475d6606b194f30bb79d271af762245 P 38a9f7e8cb214a419b9d66349741d713 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38a9f7e8cb214a419b9d66349741d713, State: Initialized, Role: FOLLOWER
I20260809 20:17:14.954941 7299 consensus_queue.cc:260] T d475d6606b194f30bb79d271af762245 P 38a9f7e8cb214a419b9d66349741d713 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:14.955899 7304 raft_consensus.cc:348] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:14.956287 7304 raft_consensus.cc:374] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:14.956385 7304 raft_consensus.cc:729] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ee8075f0e5d4e69909921c388bfeea9, State: Initialized, Role: FOLLOWER
I20260809 20:17:14.956035 7301 raft_consensus.cc:348] T d475d6606b194f30bb79d271af762245 P 7e98a98123714d3c97be8633bd03af03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:14.956570 7301 raft_consensus.cc:374] T d475d6606b194f30bb79d271af762245 P 7e98a98123714d3c97be8633bd03af03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:14.956684 7301 raft_consensus.cc:729] T d475d6606b194f30bb79d271af762245 P 7e98a98123714d3c97be8633bd03af03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e98a98123714d3c97be8633bd03af03, State: Initialized, Role: FOLLOWER
I20260809 20:17:14.957098 7304 consensus_queue.cc:260] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:14.957391 7108 heartbeater.cc:503] Master 127.5.61.62:44159 was elected leader, sending a full tablet report...
I20260809 20:17:14.957376 7301 consensus_queue.cc:260] T d475d6606b194f30bb79d271af762245 P 7e98a98123714d3c97be8633bd03af03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:14.958913 7304 ts_tablet_manager.cc:1428] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260809 20:17:14.959529 7299 ts_tablet_manager.cc:1428] T d475d6606b194f30bb79d271af762245 P 38a9f7e8cb214a419b9d66349741d713: Time spent starting tablet: real 0.016s user 0.011s sys 0.005s
I20260809 20:17:14.960462 7277 heartbeater.cc:503] Master 127.5.61.62:44159 was elected leader, sending a full tablet report...
I20260809 20:17:14.961092 7301 ts_tablet_manager.cc:1428] T d475d6606b194f30bb79d271af762245 P 7e98a98123714d3c97be8633bd03af03: Time spent starting tablet: real 0.016s user 0.013s sys 0.003s
W20260809 20:17:15.004606 6938 tablet.cc:2369] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 20:17:15.037376 7109 tablet.cc:2369] T d475d6606b194f30bb79d271af762245 P 38a9f7e8cb214a419b9d66349741d713: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 20:17:15.061563 7278 tablet.cc:2369] T d475d6606b194f30bb79d271af762245 P 7e98a98123714d3c97be8633bd03af03: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:17:15.112920 7308 raft_consensus.cc:482] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:17:15.113216 7308 raft_consensus.cc:504] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:15.114919 7308 leader_election.cc:288] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38a9f7e8cb214a419b9d66349741d713 (127.5.61.2:33415), 7e98a98123714d3c97be8633bd03af03 (127.5.61.3:37793)
I20260809 20:17:15.124068 7293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:15.115348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:49207 (local address 127.5.61.2:33415)
0809 20:17:15.115541 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:15.115547 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:15.116460 (+ 913us) server_negotiation.cc:407] Connection header received
0809 20:17:15.116675 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:15.116680 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:15.116763 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:15.116866 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:15.117787 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.118923 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.122027 (+ 3104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.123176 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.123336 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:15.123465 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:15.123734 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 20:17:15.123866 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 20:17:15.124527 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:15.115404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:36993 (local address 127.5.61.3:37793)
0809 20:17:15.115649 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:15.115656 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:15.116454 (+ 798us) server_negotiation.cc:407] Connection header received
0809 20:17:15.116702 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:15.116707 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:15.116776 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:15.116876 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:15.117749 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.118899 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.122295 (+ 3396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.123593 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.123733 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:15.123837 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:15.124199 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 20:17:15.124336 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 20:17:15.124547 7311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:15.115192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:33415 (local address 127.5.61.1:49207)
0809 20:17:15.116297 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:15.116318 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:15.116513 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:15.116900 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:15.116905 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:15.116944 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:15.117634 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:15.117647 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.119045 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.119050 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:15.121847 (+ 2797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:15.121858 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.123399 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.123403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:15.123518 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:15.123542 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 20:17:15.123700 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 20:17:15.123857 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":908,"mutex_wait_us":203,"spinlock_wait_cycles":4480,"thread_start_us":592,"threads_started":1}
I20260809 20:17:15.124660 7310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:15.115192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:37793 (local address 127.5.61.1:36993)
0809 20:17:15.116297 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:15.116325 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:15.116519 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:15.116900 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:15.116905 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:15.116939 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:15.117548 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:15.117564 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.119048 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.119052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:15.121843 (+ 2791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:15.121858 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.123841 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.123846 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:15.124016 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:15.124050 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 20:17:15.124154 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 20:17:15.124281 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1000,"mutex_wait_us":242,"spinlock_wait_cycles":81152,"thread_start_us":623,"threads_started":1}
I20260809 20:17:15.125887 7051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d475d6606b194f30bb79d271af762245" candidate_uuid: "7ee8075f0e5d4e69909921c388bfeea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38a9f7e8cb214a419b9d66349741d713" is_pre_election: true
I20260809 20:17:15.125903 7218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d475d6606b194f30bb79d271af762245" candidate_uuid: "7ee8075f0e5d4e69909921c388bfeea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e98a98123714d3c97be8633bd03af03" is_pre_election: true
I20260809 20:17:15.126503 7051 raft_consensus.cc:2454] T d475d6606b194f30bb79d271af762245 P 38a9f7e8cb214a419b9d66349741d713 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ee8075f0e5d4e69909921c388bfeea9 in term 0.
I20260809 20:17:15.126504 7218 raft_consensus.cc:2454] T d475d6606b194f30bb79d271af762245 P 7e98a98123714d3c97be8633bd03af03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ee8075f0e5d4e69909921c388bfeea9 in term 0.
I20260809 20:17:15.127199 6800 leader_election.cc:302] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38a9f7e8cb214a419b9d66349741d713, 7ee8075f0e5d4e69909921c388bfeea9; no voters:
I20260809 20:17:15.127557 7308 raft_consensus.cc:2781] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:17:15.127776 7308 raft_consensus.cc:482] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:17:15.127905 7308 raft_consensus.cc:3037] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:15.130311 7308 raft_consensus.cc:504] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:15.130903 7308 leader_election.cc:288] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9 [CANDIDATE]: Term 1 election: Requested vote from peers 38a9f7e8cb214a419b9d66349741d713 (127.5.61.2:33415), 7e98a98123714d3c97be8633bd03af03 (127.5.61.3:37793)
I20260809 20:17:15.131337 7051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d475d6606b194f30bb79d271af762245" candidate_uuid: "7ee8075f0e5d4e69909921c388bfeea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38a9f7e8cb214a419b9d66349741d713"
I20260809 20:17:15.131414 7218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d475d6606b194f30bb79d271af762245" candidate_uuid: "7ee8075f0e5d4e69909921c388bfeea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e98a98123714d3c97be8633bd03af03"
I20260809 20:17:15.131543 7051 raft_consensus.cc:3037] T d475d6606b194f30bb79d271af762245 P 38a9f7e8cb214a419b9d66349741d713 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:15.131677 7218 raft_consensus.cc:3037] T d475d6606b194f30bb79d271af762245 P 7e98a98123714d3c97be8633bd03af03 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:15.133787 7051 raft_consensus.cc:2454] T d475d6606b194f30bb79d271af762245 P 38a9f7e8cb214a419b9d66349741d713 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ee8075f0e5d4e69909921c388bfeea9 in term 1.
I20260809 20:17:15.133904 7218 raft_consensus.cc:2454] T d475d6606b194f30bb79d271af762245 P 7e98a98123714d3c97be8633bd03af03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ee8075f0e5d4e69909921c388bfeea9 in term 1.
I20260809 20:17:15.134166 6800 leader_election.cc:302] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38a9f7e8cb214a419b9d66349741d713, 7ee8075f0e5d4e69909921c388bfeea9; no voters:
I20260809 20:17:15.134445 7308 raft_consensus.cc:2781] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:15.134919 7308 raft_consensus.cc:686] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9 [term 1 LEADER]: Becoming Leader. State: Replica: 7ee8075f0e5d4e69909921c388bfeea9, State: Running, Role: LEADER
I20260809 20:17:15.135394 7308 consensus_queue.cc:237] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:15.139959 6691 catalog_manager.cc:5495] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ee8075f0e5d4e69909921c388bfeea9 (127.5.61.1). New cstate: current_term: 1 leader_uuid: "7ee8075f0e5d4e69909921c388bfeea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } health_report { overall_health: UNKNOWN } } }
I20260809 20:17:15.208931 7298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:15.203376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37538 (local address 127.5.61.1:37181)
0809 20:17:15.203632 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:15.203639 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:15.203660 (+ 21us) server_negotiation.cc:407] Connection header received
0809 20:17:15.203972 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:15.203976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:15.204070 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:15.204224 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:17:15.204602 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.205308 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.206687 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.207103 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.207240 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:15.207793 (+ 553us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:17:15.208311 (+ 518us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:17:15.208467 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:15.208604 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 20:17:15.208679 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 20:17:15.218458 7051 raft_consensus.cc:1261] T d475d6606b194f30bb79d271af762245 P 38a9f7e8cb214a419b9d66349741d713 [term 1 FOLLOWER]: Refusing update from remote peer 7ee8075f0e5d4e69909921c388bfeea9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:17:15.219632 7312 consensus_queue.cc:1035] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:17:15.219950 7218 raft_consensus.cc:1261] T d475d6606b194f30bb79d271af762245 P 7e98a98123714d3c97be8633bd03af03 [term 1 FOLLOWER]: Refusing update from remote peer 7ee8075f0e5d4e69909921c388bfeea9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:17:15.220811 7312 consensus_queue.cc:1035] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:17:15.255035 6692 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:17:15.255354 6692 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:17:15.258876 6692 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2: Sending DeleteTablet for 3 replicas of tablet d475d6606b194f30bb79d271af762245
I20260809 20:17:15.260211 6859 tablet_service.cc:1514] Processing DeleteTablet for tablet d475d6606b194f30bb79d271af762245 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:17:15 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:37534
I20260809 20:17:15.260248 7018 tablet_service.cc:1514] Processing DeleteTablet for tablet d475d6606b194f30bb79d271af762245 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:17:15 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:35960
I20260809 20:17:15.261291 7189 tablet_service.cc:1514] Processing DeleteTablet for tablet d475d6606b194f30bb79d271af762245 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:17:15 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:53206
I20260809 20:17:15.263222 6692 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:17:15.264118 6692 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:17:15.267534 7329 tablet_replica.cc:330] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9: stopping tablet replica
I20260809 20:17:15.268213 7329 raft_consensus.cc:2229] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9 [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:17:15.269433 7329 raft_consensus.cc:2258] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:15.272274 7329 ts_tablet_manager.cc:1905] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:15.272989 7188 tablet_service.cc:1467] Processing CreateTablet for tablet beb519bb97ef4a81b0c31b7f41d3877f (DEFAULT_TABLE table=test-table [id=044b3963c7d24f26952614f4a8e56737]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:15.273532 7188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet beb519bb97ef4a81b0c31b7f41d3877f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:15.273679 7017 tablet_service.cc:1467] Processing CreateTablet for tablet beb519bb97ef4a81b0c31b7f41d3877f (DEFAULT_TABLE table=test-table [id=044b3963c7d24f26952614f4a8e56737]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:15.273803 6858 tablet_service.cc:1467] Processing CreateTablet for tablet beb519bb97ef4a81b0c31b7f41d3877f (DEFAULT_TABLE table=test-table [id=044b3963c7d24f26952614f4a8e56737]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:15.274274 6858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet beb519bb97ef4a81b0c31b7f41d3877f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:15.274272 7017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet beb519bb97ef4a81b0c31b7f41d3877f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:15.280224 7301 tablet_bootstrap.cc:492] T beb519bb97ef4a81b0c31b7f41d3877f P 7e98a98123714d3c97be8633bd03af03: Bootstrap starting.
I20260809 20:17:15.280642 7304 tablet_bootstrap.cc:492] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9: Bootstrap starting.
I20260809 20:17:15.281896 7301 tablet_bootstrap.cc:654] T beb519bb97ef4a81b0c31b7f41d3877f P 7e98a98123714d3c97be8633bd03af03: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:15.282083 7299 tablet_bootstrap.cc:492] T beb519bb97ef4a81b0c31b7f41d3877f P 38a9f7e8cb214a419b9d66349741d713: Bootstrap starting.
I20260809 20:17:15.282308 7304 tablet_bootstrap.cc:654] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:15.283229 7329 ts_tablet_manager.cc:1918] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:17:15.283520 7329 log.cc:1196] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/d475d6606b194f30bb79d271af762245
I20260809 20:17:15.284178 7299 tablet_bootstrap.cc:654] T beb519bb97ef4a81b0c31b7f41d3877f P 38a9f7e8cb214a419b9d66349741d713: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:15.284436 7329 ts_tablet_manager.cc:1939] T d475d6606b194f30bb79d271af762245 P 7ee8075f0e5d4e69909921c388bfeea9: Deleting consensus metadata
I20260809 20:17:15.286550 6679 catalog_manager.cc:4841] TS 7ee8075f0e5d4e69909921c388bfeea9 (127.5.61.1:37181): tablet d475d6606b194f30bb79d271af762245 (table test-table [id=f87aee4c55cf41409e4a7fb3e3fdc946]) successfully deleted
I20260809 20:17:15.295497 7330 tablet_replica.cc:330] T d475d6606b194f30bb79d271af762245 P 38a9f7e8cb214a419b9d66349741d713: stopping tablet replica
I20260809 20:17:15.296294 7330 raft_consensus.cc:2229] T d475d6606b194f30bb79d271af762245 P 38a9f7e8cb214a419b9d66349741d713 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:17:15.296475 7299 tablet_bootstrap.cc:492] T beb519bb97ef4a81b0c31b7f41d3877f P 38a9f7e8cb214a419b9d66349741d713: No bootstrap required, opened a new log
I20260809 20:17:15.296629 7299 ts_tablet_manager.cc:1397] T beb519bb97ef4a81b0c31b7f41d3877f P 38a9f7e8cb214a419b9d66349741d713: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.006s
I20260809 20:17:15.296964 7330 raft_consensus.cc:2258] T d475d6606b194f30bb79d271af762245 P 38a9f7e8cb214a419b9d66349741d713 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:15.297243 7304 tablet_bootstrap.cc:492] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9: No bootstrap required, opened a new log
I20260809 20:17:15.297422 7304 ts_tablet_manager.cc:1397] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.006s
I20260809 20:17:15.297598 7299 raft_consensus.cc:348] T beb519bb97ef4a81b0c31b7f41d3877f P 38a9f7e8cb214a419b9d66349741d713 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:15.297864 7299 raft_consensus.cc:374] T beb519bb97ef4a81b0c31b7f41d3877f P 38a9f7e8cb214a419b9d66349741d713 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:15.297976 7299 raft_consensus.cc:729] T beb519bb97ef4a81b0c31b7f41d3877f P 38a9f7e8cb214a419b9d66349741d713 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38a9f7e8cb214a419b9d66349741d713, State: Initialized, Role: FOLLOWER
I20260809 20:17:15.298293 7299 consensus_queue.cc:260] T beb519bb97ef4a81b0c31b7f41d3877f P 38a9f7e8cb214a419b9d66349741d713 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:15.298605 7304 raft_consensus.cc:348] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:15.298929 7304 raft_consensus.cc:374] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:15.299077 7304 raft_consensus.cc:729] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ee8075f0e5d4e69909921c388bfeea9, State: Initialized, Role: FOLLOWER
I20260809 20:17:15.299496 7299 ts_tablet_manager.cc:1428] T beb519bb97ef4a81b0c31b7f41d3877f P 38a9f7e8cb214a419b9d66349741d713: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260809 20:17:15.299417 7304 consensus_queue.cc:260] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:15.300266 7304 ts_tablet_manager.cc:1428] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 20:17:15.300293 7301 tablet_bootstrap.cc:492] T beb519bb97ef4a81b0c31b7f41d3877f P 7e98a98123714d3c97be8633bd03af03: No bootstrap required, opened a new log
I20260809 20:17:15.300450 7301 ts_tablet_manager.cc:1397] T beb519bb97ef4a81b0c31b7f41d3877f P 7e98a98123714d3c97be8633bd03af03: Time spent bootstrapping tablet: real 0.020s user 0.008s sys 0.001s
I20260809 20:17:15.301889 7301 raft_consensus.cc:348] T beb519bb97ef4a81b0c31b7f41d3877f P 7e98a98123714d3c97be8633bd03af03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:15.303040 7301 raft_consensus.cc:374] T beb519bb97ef4a81b0c31b7f41d3877f P 7e98a98123714d3c97be8633bd03af03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:15.303284 7301 raft_consensus.cc:729] T beb519bb97ef4a81b0c31b7f41d3877f P 7e98a98123714d3c97be8633bd03af03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e98a98123714d3c97be8633bd03af03, State: Initialized, Role: FOLLOWER
I20260809 20:17:15.303612 7301 consensus_queue.cc:260] T beb519bb97ef4a81b0c31b7f41d3877f P 7e98a98123714d3c97be8633bd03af03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:15.304355 7330 ts_tablet_manager.cc:1905] T d475d6606b194f30bb79d271af762245 P 38a9f7e8cb214a419b9d66349741d713: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:15.304669 7301 ts_tablet_manager.cc:1428] T beb519bb97ef4a81b0c31b7f41d3877f P 7e98a98123714d3c97be8633bd03af03: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 20:17:15.308769 7331 tablet_replica.cc:330] T d475d6606b194f30bb79d271af762245 P 7e98a98123714d3c97be8633bd03af03: stopping tablet replica
I20260809 20:17:15.309209 7331 raft_consensus.cc:2229] T d475d6606b194f30bb79d271af762245 P 7e98a98123714d3c97be8633bd03af03 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:17:15.309648 7331 raft_consensus.cc:2258] T d475d6606b194f30bb79d271af762245 P 7e98a98123714d3c97be8633bd03af03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:15.311793 7331 ts_tablet_manager.cc:1905] T d475d6606b194f30bb79d271af762245 P 7e98a98123714d3c97be8633bd03af03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:15.312274 7330 ts_tablet_manager.cc:1918] T d475d6606b194f30bb79d271af762245 P 38a9f7e8cb214a419b9d66349741d713: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:17:15.312893 7330 log.cc:1196] T d475d6606b194f30bb79d271af762245 P 38a9f7e8cb214a419b9d66349741d713: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/d475d6606b194f30bb79d271af762245
I20260809 20:17:15.313598 7330 ts_tablet_manager.cc:1939] T d475d6606b194f30bb79d271af762245 P 38a9f7e8cb214a419b9d66349741d713: Deleting consensus metadata
I20260809 20:17:15.315475 6679 catalog_manager.cc:4841] TS 38a9f7e8cb214a419b9d66349741d713 (127.5.61.2:33415): tablet d475d6606b194f30bb79d271af762245 (table test-table [id=f87aee4c55cf41409e4a7fb3e3fdc946]) successfully deleted
I20260809 20:17:15.319172 7331 ts_tablet_manager.cc:1918] T d475d6606b194f30bb79d271af762245 P 7e98a98123714d3c97be8633bd03af03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:17:15.319334 7331 log.cc:1196] T d475d6606b194f30bb79d271af762245 P 7e98a98123714d3c97be8633bd03af03: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/d475d6606b194f30bb79d271af762245
I20260809 20:17:15.319876 7331 ts_tablet_manager.cc:1939] T d475d6606b194f30bb79d271af762245 P 7e98a98123714d3c97be8633bd03af03: Deleting consensus metadata
I20260809 20:17:15.321568 6674 catalog_manager.cc:4841] TS 7e98a98123714d3c97be8633bd03af03 (127.5.61.3:37793): tablet d475d6606b194f30bb79d271af762245 (table test-table [id=f87aee4c55cf41409e4a7fb3e3fdc946]) successfully deleted
I20260809 20:17:15.362708 7322 raft_consensus.cc:482] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:17:15.362974 7322 raft_consensus.cc:504] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:15.363965 7322 leader_election.cc:288] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38a9f7e8cb214a419b9d66349741d713 (127.5.61.2:33415), 7e98a98123714d3c97be8633bd03af03 (127.5.61.3:37793)
I20260809 20:17:15.364504 7218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb519bb97ef4a81b0c31b7f41d3877f" candidate_uuid: "7ee8075f0e5d4e69909921c388bfeea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e98a98123714d3c97be8633bd03af03" is_pre_election: true
I20260809 20:17:15.364519 7051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb519bb97ef4a81b0c31b7f41d3877f" candidate_uuid: "7ee8075f0e5d4e69909921c388bfeea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38a9f7e8cb214a419b9d66349741d713" is_pre_election: true
I20260809 20:17:15.364776 7051 raft_consensus.cc:2454] T beb519bb97ef4a81b0c31b7f41d3877f P 38a9f7e8cb214a419b9d66349741d713 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ee8075f0e5d4e69909921c388bfeea9 in term 0.
I20260809 20:17:15.364791 7218 raft_consensus.cc:2454] T beb519bb97ef4a81b0c31b7f41d3877f P 7e98a98123714d3c97be8633bd03af03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ee8075f0e5d4e69909921c388bfeea9 in term 0.
I20260809 20:17:15.365254 6795 leader_election.cc:302] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e98a98123714d3c97be8633bd03af03, 7ee8075f0e5d4e69909921c388bfeea9; no voters:
I20260809 20:17:15.365559 7322 raft_consensus.cc:2781] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:17:15.365711 7322 raft_consensus.cc:482] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:17:15.365825 7322 raft_consensus.cc:3037] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:15.369145 7322 raft_consensus.cc:504] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:15.369832 7322 leader_election.cc:288] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9 [CANDIDATE]: Term 1 election: Requested vote from peers 38a9f7e8cb214a419b9d66349741d713 (127.5.61.2:33415), 7e98a98123714d3c97be8633bd03af03 (127.5.61.3:37793)
I20260809 20:17:15.370325 7218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb519bb97ef4a81b0c31b7f41d3877f" candidate_uuid: "7ee8075f0e5d4e69909921c388bfeea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e98a98123714d3c97be8633bd03af03"
I20260809 20:17:15.370343 7051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beb519bb97ef4a81b0c31b7f41d3877f" candidate_uuid: "7ee8075f0e5d4e69909921c388bfeea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38a9f7e8cb214a419b9d66349741d713"
I20260809 20:17:15.370554 7218 raft_consensus.cc:3037] T beb519bb97ef4a81b0c31b7f41d3877f P 7e98a98123714d3c97be8633bd03af03 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:15.370565 7051 raft_consensus.cc:3037] T beb519bb97ef4a81b0c31b7f41d3877f P 38a9f7e8cb214a419b9d66349741d713 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:15.372844 7218 raft_consensus.cc:2454] T beb519bb97ef4a81b0c31b7f41d3877f P 7e98a98123714d3c97be8633bd03af03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ee8075f0e5d4e69909921c388bfeea9 in term 1.
I20260809 20:17:15.373279 6795 leader_election.cc:302] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e98a98123714d3c97be8633bd03af03, 7ee8075f0e5d4e69909921c388bfeea9; no voters:
I20260809 20:17:15.373624 7322 raft_consensus.cc:2781] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:15.373641 7051 raft_consensus.cc:2454] T beb519bb97ef4a81b0c31b7f41d3877f P 38a9f7e8cb214a419b9d66349741d713 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ee8075f0e5d4e69909921c388bfeea9 in term 1.
I20260809 20:17:15.374164 7322 raft_consensus.cc:686] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9 [term 1 LEADER]: Becoming Leader. State: Replica: 7ee8075f0e5d4e69909921c388bfeea9, State: Running, Role: LEADER
I20260809 20:17:15.374540 7322 consensus_queue.cc:237] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:15.377358 6691 catalog_manager.cc:5495] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ee8075f0e5d4e69909921c388bfeea9 (127.5.61.1). New cstate: current_term: 1 leader_uuid: "7ee8075f0e5d4e69909921c388bfeea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } health_report { overall_health: UNKNOWN } } }
Aug 09 20:17:15 dist-test-slave-6k22 krb5kdc[6640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306633, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:15.421845 7341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:15.402016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35702 (local address 127.5.61.62:44159)
0809 20:17:15.402780 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:15.402790 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:15.403035 (+ 245us) server_negotiation.cc:407] Connection header received
0809 20:17:15.403247 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:15.403259 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:15.403313 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:15.403422 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:15.404702 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.405520 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.406783 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.407209 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.407352 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:15.415882 (+ 8530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:15.415888 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:15.415891 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:15.415922 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:15.418906 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:15.419796 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:15.419800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:15.419802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:15.419926 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:15.420362 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:15.420366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:15.420368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:15.420642 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:15.420751 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:15.421250 (+ 499us) server_negotiation.cc:299] Negotiation successful
0809 20:17:15.421584 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":141,"server-negotiator.queue_time_us":572,"thread_start_us":392,"threads_started":1}
I20260809 20:17:15.421891 7342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:15.402098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:44159 (local address 127.0.0.1:35702)
0809 20:17:15.402931 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:15.402946 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:15.403117 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:15.403457 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:15.403464 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:15.404259 (+ 795us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:15.404535 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:15.404545 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.405651 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.405658 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:15.406636 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:15.406648 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.407396 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.407402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:15.407622 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:15.412842 (+ 5220us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:15.412860 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:15.415628 (+ 2768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:15.419045 (+ 3417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:15.419058 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:15.419066 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:15.419682 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:15.420035 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:15.420041 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:15.420045 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:15.420243 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:15.420780 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:15.420789 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:15.420983 (+ 194us) client_negotiation.cc:769] Sending connection context
0809 20:17:15.421155 (+ 172us) client_negotiation.cc:240] Negotiation successful
0809 20:17:15.421448 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":540,"threads_started":1}
I20260809 20:17:15.427129 6691 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:35702:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 20:17:15.428354 6691 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:17:15.435328 6858 tablet_service.cc:1467] Processing CreateTablet for tablet 2dd220e0dc234568b4a8e3833c8a6a48 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d7281fc7ea0347a087ca0f9977d8355e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 20:17:15.435436 7018 tablet_service.cc:1467] Processing CreateTablet for tablet 2dd220e0dc234568b4a8e3833c8a6a48 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d7281fc7ea0347a087ca0f9977d8355e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 20:17:15.435962 6858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dd220e0dc234568b4a8e3833c8a6a48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:15.436048 7018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dd220e0dc234568b4a8e3833c8a6a48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:15.439173 7189 tablet_service.cc:1467] Processing CreateTablet for tablet 2dd220e0dc234568b4a8e3833c8a6a48 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d7281fc7ea0347a087ca0f9977d8355e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 20:17:15.439849 7189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dd220e0dc234568b4a8e3833c8a6a48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:15.443706 7299 tablet_bootstrap.cc:492] T 2dd220e0dc234568b4a8e3833c8a6a48 P 38a9f7e8cb214a419b9d66349741d713: Bootstrap starting.
I20260809 20:17:15.445492 7299 tablet_bootstrap.cc:654] T 2dd220e0dc234568b4a8e3833c8a6a48 P 38a9f7e8cb214a419b9d66349741d713: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:15.446054 7304 tablet_bootstrap.cc:492] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7ee8075f0e5d4e69909921c388bfeea9: Bootstrap starting.
I20260809 20:17:15.446589 7301 tablet_bootstrap.cc:492] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03: Bootstrap starting.
I20260809 20:17:15.447997 7304 tablet_bootstrap.cc:654] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7ee8075f0e5d4e69909921c388bfeea9: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:15.448339 7301 tablet_bootstrap.cc:654] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:15.461885 7299 tablet_bootstrap.cc:492] T 2dd220e0dc234568b4a8e3833c8a6a48 P 38a9f7e8cb214a419b9d66349741d713: No bootstrap required, opened a new log
I20260809 20:17:15.462225 7299 ts_tablet_manager.cc:1397] T 2dd220e0dc234568b4a8e3833c8a6a48 P 38a9f7e8cb214a419b9d66349741d713: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.009s
I20260809 20:17:15.463064 7299 raft_consensus.cc:348] T 2dd220e0dc234568b4a8e3833c8a6a48 P 38a9f7e8cb214a419b9d66349741d713 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:15.463400 7299 raft_consensus.cc:374] T 2dd220e0dc234568b4a8e3833c8a6a48 P 38a9f7e8cb214a419b9d66349741d713 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:15.463500 7299 raft_consensus.cc:729] T 2dd220e0dc234568b4a8e3833c8a6a48 P 38a9f7e8cb214a419b9d66349741d713 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38a9f7e8cb214a419b9d66349741d713, State: Initialized, Role: FOLLOWER
I20260809 20:17:15.463892 7304 tablet_bootstrap.cc:492] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7ee8075f0e5d4e69909921c388bfeea9: No bootstrap required, opened a new log
I20260809 20:17:15.463819 7299 consensus_queue.cc:260] T 2dd220e0dc234568b4a8e3833c8a6a48 P 38a9f7e8cb214a419b9d66349741d713 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:15.464179 7304 ts_tablet_manager.cc:1397] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7ee8075f0e5d4e69909921c388bfeea9: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.005s
I20260809 20:17:15.464489 7299 ts_tablet_manager.cc:1428] T 2dd220e0dc234568b4a8e3833c8a6a48 P 38a9f7e8cb214a419b9d66349741d713: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260809 20:17:15.464388 7307 tablet_replica.cc:439] T 2dd220e0dc234568b4a8e3833c8a6a48 P 38a9f7e8cb214a419b9d66349741d713: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } } }
I20260809 20:17:15.464808 7307 tablet_replica.cc:442] T 2dd220e0dc234568b4a8e3833c8a6a48 P 38a9f7e8cb214a419b9d66349741d713: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 20:17:15.465271 7301 tablet_bootstrap.cc:492] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03: No bootstrap required, opened a new log
I20260809 20:17:15.465442 7301 ts_tablet_manager.cc:1397] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.008s
I20260809 20:17:15.465221 7304 raft_consensus.cc:348] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7ee8075f0e5d4e69909921c388bfeea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:15.465896 7304 raft_consensus.cc:374] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7ee8075f0e5d4e69909921c388bfeea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:15.465993 7304 raft_consensus.cc:729] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7ee8075f0e5d4e69909921c388bfeea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ee8075f0e5d4e69909921c388bfeea9, State: Initialized, Role: FOLLOWER
I20260809 20:17:15.466207 7301 raft_consensus.cc:348] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:15.466447 7301 raft_consensus.cc:374] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:15.466295 7304 consensus_queue.cc:260] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7ee8075f0e5d4e69909921c388bfeea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:15.466516 7301 raft_consensus.cc:729] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e98a98123714d3c97be8633bd03af03, State: Initialized, Role: FOLLOWER
I20260809 20:17:15.466755 7322 tablet_replica.cc:439] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7ee8075f0e5d4e69909921c388bfeea9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } } }
I20260809 20:17:15.466790 7301 consensus_queue.cc:260] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:15.467097 7322 tablet_replica.cc:442] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7ee8075f0e5d4e69909921c388bfeea9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 20:17:15.467245 7309 tablet_replica.cc:439] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } } }
I20260809 20:17:15.467543 7309 tablet_replica.cc:442] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 20:17:15.469537 7301 ts_tablet_manager.cc:1428] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 20:17:15.470603 7304 ts_tablet_manager.cc:1428] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7ee8075f0e5d4e69909921c388bfeea9: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260809 20:17:15.509112 7309 raft_consensus.cc:482] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:17:15.509375 7309 raft_consensus.cc:504] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:15.511219 7309 leader_election.cc:288] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ee8075f0e5d4e69909921c388bfeea9 (127.5.61.1:37181), 38a9f7e8cb214a419b9d66349741d713 (127.5.61.2:33415)
I20260809 20:17:15.520126 7293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:15.512523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:48051 (local address 127.5.61.2:33415)
0809 20:17:15.512676 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:15.512681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:15.513797 (+ 1116us) server_negotiation.cc:407] Connection header received
0809 20:17:15.514046 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:15.514055 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:15.514139 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:15.514235 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:15.514801 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.515809 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.517808 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.519079 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.519226 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:15.519338 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:15.519637 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 20:17:15.519798 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 20:17:15.520466 7298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:15.511745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:43807 (local address 127.5.61.1:37181)
0809 20:17:15.511999 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:15.512005 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:15.512285 (+ 280us) server_negotiation.cc:407] Connection header received
0809 20:17:15.512600 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:15.512609 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:15.512751 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:15.512830 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:15.513960 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.515170 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.517871 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.519117 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.519281 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:15.519371 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:15.519994 (+ 623us) server_negotiation.cc:299] Negotiation successful
0809 20:17:15.520157 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 20:17:15.520169 7347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:15.512391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:33415 (local address 127.5.61.3:48051)
0809 20:17:15.513661 (+ 1270us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:15.513679 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:15.513852 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:15.514262 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:15.514266 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:15.514291 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:15.514629 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:15.514639 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.515943 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.515949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:15.517644 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:15.517657 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.519260 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.519268 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:15.519438 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:15.519476 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 20:17:15.519608 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 20:17:15.519783 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1142,"mutex_wait_us":29,"thread_start_us":936,"threads_started":1}
I20260809 20:17:15.521378 7346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:15.511623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37181 (local address 127.5.61.3:43807)
0809 20:17:15.512167 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:15.512194 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:15.512404 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:15.513179 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:15.513187 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:15.513227 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:15.513683 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:15.513696 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.515367 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.515372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:15.517664 (+ 2292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:15.517679 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.519517 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.519521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:15.519804 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:15.519836 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 20:17:15.520715 (+ 879us) client_negotiation.cc:240] Negotiation successful
0809 20:17:15.520809 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":49,"thread_start_us":387,"threads_started":1}
I20260809 20:17:15.521718 7051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dd220e0dc234568b4a8e3833c8a6a48" candidate_uuid: "7e98a98123714d3c97be8633bd03af03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38a9f7e8cb214a419b9d66349741d713" is_pre_election: true
I20260809 20:17:15.522022 7051 raft_consensus.cc:2454] T 2dd220e0dc234568b4a8e3833c8a6a48 P 38a9f7e8cb214a419b9d66349741d713 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e98a98123714d3c97be8633bd03af03 in term 0.
I20260809 20:17:15.522464 7132 leader_election.cc:302] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38a9f7e8cb214a419b9d66349741d713, 7e98a98123714d3c97be8633bd03af03; no voters:
I20260809 20:17:15.522527 6881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dd220e0dc234568b4a8e3833c8a6a48" candidate_uuid: "7e98a98123714d3c97be8633bd03af03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ee8075f0e5d4e69909921c388bfeea9" is_pre_election: true
I20260809 20:17:15.522866 7309 raft_consensus.cc:2781] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:17:15.522989 7309 raft_consensus.cc:482] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:17:15.522989 6881 raft_consensus.cc:2454] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7ee8075f0e5d4e69909921c388bfeea9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e98a98123714d3c97be8633bd03af03 in term 0.
I20260809 20:17:15.523097 7309 raft_consensus.cc:3037] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:15.526419 7309 raft_consensus.cc:504] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:15.527091 7309 leader_election.cc:288] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03 [CANDIDATE]: Term 1 election: Requested vote from peers 7ee8075f0e5d4e69909921c388bfeea9 (127.5.61.1:37181), 38a9f7e8cb214a419b9d66349741d713 (127.5.61.2:33415)
I20260809 20:17:15.527667 6881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dd220e0dc234568b4a8e3833c8a6a48" candidate_uuid: "7e98a98123714d3c97be8633bd03af03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ee8075f0e5d4e69909921c388bfeea9"
I20260809 20:17:15.527724 7051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dd220e0dc234568b4a8e3833c8a6a48" candidate_uuid: "7e98a98123714d3c97be8633bd03af03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38a9f7e8cb214a419b9d66349741d713"
I20260809 20:17:15.527940 6881 raft_consensus.cc:3037] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7ee8075f0e5d4e69909921c388bfeea9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:15.527992 7051 raft_consensus.cc:3037] T 2dd220e0dc234568b4a8e3833c8a6a48 P 38a9f7e8cb214a419b9d66349741d713 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:15.530922 7051 raft_consensus.cc:2454] T 2dd220e0dc234568b4a8e3833c8a6a48 P 38a9f7e8cb214a419b9d66349741d713 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e98a98123714d3c97be8633bd03af03 in term 1.
I20260809 20:17:15.531267 6881 raft_consensus.cc:2454] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7ee8075f0e5d4e69909921c388bfeea9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e98a98123714d3c97be8633bd03af03 in term 1.
I20260809 20:17:15.531327 7132 leader_election.cc:302] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38a9f7e8cb214a419b9d66349741d713, 7e98a98123714d3c97be8633bd03af03; no voters:
I20260809 20:17:15.531656 7309 raft_consensus.cc:2781] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:15.532122 7309 raft_consensus.cc:686] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03 [term 1 LEADER]: Becoming Leader. State: Replica: 7e98a98123714d3c97be8633bd03af03, State: Running, Role: LEADER
I20260809 20:17:15.532542 7309 consensus_queue.cc:237] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } }
I20260809 20:17:15.534958 7348 tablet_replica.cc:439] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03: TxnStatusTablet state changed. Reason: New leader 7e98a98123714d3c97be8633bd03af03. Latest consensus state: current_term: 1 leader_uuid: "7e98a98123714d3c97be8633bd03af03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } } }
I20260809 20:17:15.535409 7348 tablet_replica.cc:442] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03: This TxnStatusTablet replica's current role is: LEADER
I20260809 20:17:15.536279 7352 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 20:17:15.536578 7352 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 20:17:15.536849 6691 catalog_manager.cc:5495] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e98a98123714d3c97be8633bd03af03 (127.5.61.3). New cstate: current_term: 1 leader_uuid: "7e98a98123714d3c97be8633bd03af03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } health_report { overall_health: HEALTHY } } }
I20260809 20:17:15.556797 7342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:15.550912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:37793 (local address 127.0.0.1:53212)
0809 20:17:15.551116 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:15.551137 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:15.551306 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:15.551829 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:15.551833 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:15.551845 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 20:17:15.552038 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:15.552047 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.553194 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.553200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:15.554346 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:15.554359 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.555108 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.555113 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:15.555314 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:15.555363 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 20:17:15.556190 (+ 827us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 20:17:15.556210 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 20:17:15.556373 (+ 163us) client_negotiation.cc:240] Negotiation successful
0809 20:17:15.556524 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 20:17:15.556825 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:15.550972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53212 (local address 127.5.61.3:37793)
0809 20:17:15.551140 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:15.551146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:15.551215 (+ 69us) server_negotiation.cc:407] Connection header received
0809 20:17:15.551508 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:15.551516 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:15.551665 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:15.551817 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:17:15.552170 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.553071 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.554488 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.554938 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.555084 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:15.555604 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:17:15.556073 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:17:15.556224 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:15.556401 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 20:17:15.556482 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 20:17:15.561448 7051 raft_consensus.cc:1261] T 2dd220e0dc234568b4a8e3833c8a6a48 P 38a9f7e8cb214a419b9d66349741d713 [term 1 FOLLOWER]: Refusing update from remote peer 7e98a98123714d3c97be8633bd03af03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:17:15.561609 6881 raft_consensus.cc:1261] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7ee8075f0e5d4e69909921c388bfeea9 [term 1 FOLLOWER]: Refusing update from remote peer 7e98a98123714d3c97be8633bd03af03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:17:15.562363 7348 consensus_queue.cc:1035] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:17:15.563097 7309 consensus_queue.cc:1035] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:17:15.568841 7307 tablet_replica.cc:439] T 2dd220e0dc234568b4a8e3833c8a6a48 P 38a9f7e8cb214a419b9d66349741d713: TxnStatusTablet state changed. Reason: New leader 7e98a98123714d3c97be8633bd03af03. Latest consensus state: current_term: 1 leader_uuid: "7e98a98123714d3c97be8633bd03af03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } } }
I20260809 20:17:15.569109 7307 tablet_replica.cc:442] T 2dd220e0dc234568b4a8e3833c8a6a48 P 38a9f7e8cb214a419b9d66349741d713: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 20:17:15.570001 7309 tablet_replica.cc:439] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7e98a98123714d3c97be8633bd03af03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } } }
I20260809 20:17:15.570271 7309 tablet_replica.cc:442] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03: This TxnStatusTablet replica's current role is: LEADER
I20260809 20:17:15.575619 7317 mvcc.cc:205] Tried to move back new op lower bound from 7316711979251511296 to 7316711979145834496. Current Snapshot: MvccSnapshot[applied={T|T < 7316711979251511296}]
I20260809 20:17:15.577401 7307 tablet_replica.cc:439] T 2dd220e0dc234568b4a8e3833c8a6a48 P 38a9f7e8cb214a419b9d66349741d713: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7e98a98123714d3c97be8633bd03af03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } } }
I20260809 20:17:15.577603 7307 tablet_replica.cc:442] T 2dd220e0dc234568b4a8e3833c8a6a48 P 38a9f7e8cb214a419b9d66349741d713: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 20:17:15.582436 7322 tablet_replica.cc:439] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7ee8075f0e5d4e69909921c388bfeea9: TxnStatusTablet state changed. Reason: New leader 7e98a98123714d3c97be8633bd03af03. Latest consensus state: current_term: 1 leader_uuid: "7e98a98123714d3c97be8633bd03af03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } } }
I20260809 20:17:15.582813 7322 tablet_replica.cc:442] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7ee8075f0e5d4e69909921c388bfeea9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 20:17:15.584443 7348 tablet_replica.cc:439] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7e98a98123714d3c97be8633bd03af03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } } }
I20260809 20:17:15.584700 7348 tablet_replica.cc:442] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7e98a98123714d3c97be8633bd03af03: This TxnStatusTablet replica's current role is: LEADER
I20260809 20:17:15.585964 7316 mvcc.cc:205] Tried to move back new op lower bound from 7316711979251511296 to 7316711979145834496. Current Snapshot: MvccSnapshot[applied={T|T < 7316711979251511296}]
I20260809 20:17:15.586232 7322 tablet_replica.cc:439] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7ee8075f0e5d4e69909921c388bfeea9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7e98a98123714d3c97be8633bd03af03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ee8075f0e5d4e69909921c388bfeea9" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37181 } } peers { permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 } } peers { permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 } } }
I20260809 20:17:15.586496 7322 tablet_replica.cc:442] T 2dd220e0dc234568b4a8e3833c8a6a48 P 7ee8075f0e5d4e69909921c388bfeea9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 20:17:15.609233 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:15.596191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:42817 (local address 127.5.61.3:37793)
0809 20:17:15.596425 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:15.596434 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:15.604240 (+ 7806us) server_negotiation.cc:407] Connection header received
0809 20:17:15.604493 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:15.604497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:15.604560 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:15.604655 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:17:15.605197 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.605907 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.607379 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.607935 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.608093 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:15.608543 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:17:15.608633 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:17:15.608728 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:15.608862 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 20:17:15.608937 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 20:17:15.609660 7360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:15.596116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:37793 (local address 127.5.61.1:42817)
0809 20:17:15.604134 (+ 8018us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:15.604151 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:15.604326 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:15.604682 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:15.604687 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:15.604700 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 20:17:15.605052 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:15.605061 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.606036 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.606042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:15.607226 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:15.607246 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.608135 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.608147 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:15.608310 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:15.608362 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 20:17:15.608730 (+ 368us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 20:17:15.608742 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 20:17:15.608841 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 20:17:15.608989 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7872,"thread_start_us":7709,"threads_started":1}
I20260809 20:17:15.630115 7218 raft_consensus.cc:1261] T beb519bb97ef4a81b0c31b7f41d3877f P 7e98a98123714d3c97be8633bd03af03 [term 1 FOLLOWER]: Refusing update from remote peer 7ee8075f0e5d4e69909921c388bfeea9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:17:15.631006 7312 consensus_queue.cc:1035] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e98a98123714d3c97be8633bd03af03" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 37793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:17:15.635699 7051 raft_consensus.cc:1261] T beb519bb97ef4a81b0c31b7f41d3877f P 38a9f7e8cb214a419b9d66349741d713 [term 1 FOLLOWER]: Refusing update from remote peer 7ee8075f0e5d4e69909921c388bfeea9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:17:15.636559 7312 consensus_queue.cc:1035] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38a9f7e8cb214a419b9d66349741d713" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:17:15.647992 7318 mvcc.cc:205] Tried to move back new op lower bound from 7316711979531268096 to 7316711978496946176. Current Snapshot: MvccSnapshot[applied={T|T < 7316711979531268096}]
I20260809 20:17:15.702612 7298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:15.679081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:38565 (local address 127.5.61.1:37181)
0809 20:17:15.679285 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:15.679292 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:15.695120 (+ 15828us) server_negotiation.cc:407] Connection header received
0809 20:17:15.695311 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:15.695316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:15.695423 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:15.695535 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:17:15.696220 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.697435 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.700228 (+ 2793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.700966 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.701103 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:15.701742 (+ 639us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:17:15.701886 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:17:15.701989 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:15.702198 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 20:17:15.702304 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 20:17:15.703918 7365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:15.679099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37181 (local address 127.5.61.3:38565)
0809 20:17:15.694964 (+ 15865us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:15.694991 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:15.695162 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:15.695601 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:15.695606 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:15.695622 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 20:17:15.696007 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:15.696020 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.697662 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.697669 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:15.700042 (+ 2373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:15.700063 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.701209 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.701219 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:15.701411 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:15.701511 (+ 100us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 20:17:15.702016 (+ 505us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 20:17:15.702030 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 20:17:15.702125 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 20:17:15.702284 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15634,"thread_start_us":15471,"threads_started":1}
I20260809 20:17:15.753000 6691 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:17:15.753300 6691 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:17:15.757678 6691 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 972bc05eeaa441e48fb9d9f10e92b4a2: Sending DeleteTablet for 3 replicas of tablet beb519bb97ef4a81b0c31b7f41d3877f
I20260809 20:17:15.759176 7189 tablet_service.cc:1514] Processing DeleteTablet for tablet beb519bb97ef4a81b0c31b7f41d3877f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:17:15 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:53206
I20260809 20:17:15.759529 7331 tablet_replica.cc:330] T beb519bb97ef4a81b0c31b7f41d3877f P 7e98a98123714d3c97be8633bd03af03: stopping tablet replica
I20260809 20:17:15.759917 7331 raft_consensus.cc:2229] T beb519bb97ef4a81b0c31b7f41d3877f P 7e98a98123714d3c97be8633bd03af03 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:17:15.760157 7331 raft_consensus.cc:2258] T beb519bb97ef4a81b0c31b7f41d3877f P 7e98a98123714d3c97be8633bd03af03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:15.760965 6858 tablet_service.cc:1514] Processing DeleteTablet for tablet beb519bb97ef4a81b0c31b7f41d3877f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:17:15 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:37534
I20260809 20:17:15.761286 7329 tablet_replica.cc:330] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9: stopping tablet replica
I20260809 20:17:15.761557 7329 raft_consensus.cc:2229] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9 [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:17:15.761695 7331 ts_tablet_manager.cc:1905] T beb519bb97ef4a81b0c31b7f41d3877f P 7e98a98123714d3c97be8633bd03af03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:15.762004 7329 raft_consensus.cc:2258] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:15.762212 7018 tablet_service.cc:1514] Processing DeleteTablet for tablet beb519bb97ef4a81b0c31b7f41d3877f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:17:15 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:35960
I20260809 20:17:15.762799 7330 tablet_replica.cc:330] T beb519bb97ef4a81b0c31b7f41d3877f P 38a9f7e8cb214a419b9d66349741d713: stopping tablet replica
I20260809 20:17:15.763046 7330 raft_consensus.cc:2229] T beb519bb97ef4a81b0c31b7f41d3877f P 38a9f7e8cb214a419b9d66349741d713 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:17:15.763293 7330 raft_consensus.cc:2258] T beb519bb97ef4a81b0c31b7f41d3877f P 38a9f7e8cb214a419b9d66349741d713 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:15.764452 7329 ts_tablet_manager.cc:1905] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:15.764668 7330 ts_tablet_manager.cc:1905] T beb519bb97ef4a81b0c31b7f41d3877f P 38a9f7e8cb214a419b9d66349741d713: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:15.777209 7331 ts_tablet_manager.cc:1918] T beb519bb97ef4a81b0c31b7f41d3877f P 7e98a98123714d3c97be8633bd03af03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 20:17:15.777405 7331 log.cc:1196] T beb519bb97ef4a81b0c31b7f41d3877f P 7e98a98123714d3c97be8633bd03af03: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/beb519bb97ef4a81b0c31b7f41d3877f
I20260809 20:17:15.777786 7331 ts_tablet_manager.cc:1939] T beb519bb97ef4a81b0c31b7f41d3877f P 7e98a98123714d3c97be8633bd03af03: Deleting consensus metadata
I20260809 20:17:15.778837 7330 ts_tablet_manager.cc:1918] T beb519bb97ef4a81b0c31b7f41d3877f P 38a9f7e8cb214a419b9d66349741d713: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 20:17:15.778999 7330 log.cc:1196] T beb519bb97ef4a81b0c31b7f41d3877f P 38a9f7e8cb214a419b9d66349741d713: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/beb519bb97ef4a81b0c31b7f41d3877f
I20260809 20:17:15.779074 6674 catalog_manager.cc:4841] TS 7e98a98123714d3c97be8633bd03af03 (127.5.61.3:37793): tablet beb519bb97ef4a81b0c31b7f41d3877f (table test-table [id=044b3963c7d24f26952614f4a8e56737]) successfully deleted
I20260809 20:17:15.779465 7330 ts_tablet_manager.cc:1939] T beb519bb97ef4a81b0c31b7f41d3877f P 38a9f7e8cb214a419b9d66349741d713: Deleting consensus metadata
I20260809 20:17:15.780764 6679 catalog_manager.cc:4841] TS 38a9f7e8cb214a419b9d66349741d713 (127.5.61.2:33415): tablet beb519bb97ef4a81b0c31b7f41d3877f (table test-table [id=044b3963c7d24f26952614f4a8e56737]) successfully deleted
I20260809 20:17:15.782541 7329 ts_tablet_manager.cc:1918] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 20:17:15.782833 7329 log.cc:1196] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/beb519bb97ef4a81b0c31b7f41d3877f
I20260809 20:17:15.783517 7329 ts_tablet_manager.cc:1939] T beb519bb97ef4a81b0c31b7f41d3877f P 7ee8075f0e5d4e69909921c388bfeea9: Deleting consensus metadata
I20260809 20:17:15.785209 6679 catalog_manager.cc:4841] TS 7ee8075f0e5d4e69909921c388bfeea9 (127.5.61.1:37181): tablet beb519bb97ef4a81b0c31b7f41d3877f (table test-table [id=044b3963c7d24f26952614f4a8e56737]) successfully deleted
Aug 09 20:17:15 dist-test-slave-6k22 krb5kdc[6640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306634, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.61.1@KRBTEST.COM
I20260809 20:17:15.818293 7298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:15.792989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37554 (local address 127.5.61.1:37181)
0809 20:17:15.793271 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:15.793278 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:15.793881 (+ 603us) server_negotiation.cc:407] Connection header received
0809 20:17:15.794197 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:15.794204 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:15.794297 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:15.794388 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:15.796966 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.798334 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.800869 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.801710 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.801829 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:15.808644 (+ 6815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:15.808663 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:15.808683 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:15.808729 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:15.813744 (+ 5015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:15.814875 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:15.814887 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:15.814890 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:15.815091 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:15.815693 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:15.815730 (+ 37us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:15.815734 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:15.816253 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:15.816864 (+ 611us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:15.817545 (+ 681us) server_negotiation.cc:299] Negotiation successful
0809 20:17:15.817821 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 20:17:15.820458 6932 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37554
I20260809 20:17:15.860387 7341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:15.832972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35712 (local address 127.5.61.62:44159)
0809 20:17:15.833293 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:15.833305 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:15.840122 (+ 6817us) server_negotiation.cc:407] Connection header received
0809 20:17:15.840283 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:15.840289 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:15.840390 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:15.840509 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:15.843126 (+ 2617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.844550 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.846761 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.847670 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.847840 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:15.852024 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:15.852032 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:15.852038 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:15.852102 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:15.856564 (+ 4462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:15.857544 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:15.857551 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:15.857555 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:15.857762 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:15.858264 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:15.858271 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:15.858275 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:15.858741 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:15.859029 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:15.859413 (+ 384us) server_negotiation.cc:299] Negotiation successful
0809 20:17:15.859984 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
W20260809 20:17:15.861495 6691 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35712
I20260809 20:17:15.863862 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 6771
I20260809 20:17:15.882020 7298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:15.865503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37558 (local address 127.5.61.1:37181)
0809 20:17:15.865737 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:15.865746 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:15.868082 (+ 2336us) server_negotiation.cc:407] Connection header received
0809 20:17:15.868363 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:15.868369 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:15.868447 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:15.868557 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:15.870365 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.870981 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.872765 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.873109 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.873195 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:15.876560 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:15.876565 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:15.876568 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:15.876598 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:15.879175 (+ 2577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:15.880268 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:15.880273 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:15.880276 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:15.880458 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:15.880873 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:15.880876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:15.880878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:15.881135 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:15.881307 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:15.881656 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 20:17:15.881798 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 20:17:15.882743 6932 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37558
W20260809 20:17:15.883301 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 6771: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 20:17:15.883507 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 6771
I20260809 20:17:15.914115 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 6942
Aug 09 20:17:15 dist-test-slave-6k22 krb5kdc[6640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306634, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.61.2@KRBTEST.COM
I20260809 20:17:15.930770 7293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:15.915178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35966 (local address 127.5.61.2:33415)
0809 20:17:15.915407 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:15.915415 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:15.915442 (+ 27us) server_negotiation.cc:407] Connection header received
0809 20:17:15.915520 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:15.915527 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:15.915757 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:15.915893 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:15.917003 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.918190 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.919449 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.920264 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.920445 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:15.924949 (+ 4504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:15.924962 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:15.924975 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:15.925004 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:15.927934 (+ 2930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:15.928819 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:15.928825 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:15.928827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:15.928954 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:15.929407 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:15.929410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:15.929412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:15.929716 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:15.929934 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:15.930371 (+ 437us) server_negotiation.cc:299] Negotiation successful
0809 20:17:15.930538 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260809 20:17:15.931495 7102 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35966
W20260809 20:17:15.932209 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 6942: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 20:17:15.932379 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 6942
I20260809 20:17:15.962647 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 7112
Aug 09 20:17:15 dist-test-slave-6k22 krb5kdc[6640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306634, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.61.3@KRBTEST.COM
I20260809 20:17:15.978736 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:15.963488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53214 (local address 127.5.61.3:37793)
0809 20:17:15.963720 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:15.963727 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:15.963749 (+ 22us) server_negotiation.cc:407] Connection header received
0809 20:17:15.963891 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:15.963898 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:15.963985 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:15.964098 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:15.965361 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.966172 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.967457 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:15.968000 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:15.968143 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:15.972637 (+ 4494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:15.972648 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:15.972662 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:15.972694 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:15.975976 (+ 3282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:15.976655 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:15.976662 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:15.976665 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:15.976838 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:15.977222 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:15.977227 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:15.977232 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:15.977594 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:15.977908 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:15.978122 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 20:17:15.978354 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
W20260809 20:17:15.979868 7271 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53214
W20260809 20:17:15.980630 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 7112: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 20:17:15.980832 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 7112
I20260809 20:17:16.006896 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 6654
I20260809 20:17:16.008219 6736 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:16.121073 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 6654
2026-08-09T20:17:16Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4755 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:17:16 dist-test-slave-6k22 krb5kdc[7376](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:17:16 dist-test-slave-6k22 krb5kdc[7376](info): set up 2 sockets
Aug 09 20:17:16 dist-test-slave-6k22 krb5kdc[7376](info): commencing operation
krb5kdc: starting...
W20260809 20:17:17.282186 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:17:17 dist-test-slave-6k22 krb5kdc[7376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:17:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:17:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:17:17.479403 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:46769
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:40081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:46769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:17.804354 7391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:17.804672 7391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:17.804857 7391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:17.812021 7391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:17:17.812155 7391 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 20:17:17.812237 7391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:17.812492 7391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:17.812589 7391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:17:17.812733 7391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:17:17.822043 7391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:40081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:46769
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:46769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:17.823287 7391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:17.824612 7391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:17.836473 7397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:17.836555 7398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:17.836434 7401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:17.837182 7391 server_base.cc:1034] running on GCE node
I20260809 20:17:17.838033 7391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:17.839354 7391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:17.840608 7391 hybrid_clock.cc:648] HybridClock initialized: now 1786306637840556 us; error 44 us; skew 500 ppm
I20260809 20:17:17.840937 7391 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:17 dist-test-slave-6k22 krb5kdc[7376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306637, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:17.844038 7391 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:17:17.845461 7391 webserver.cc:460] Webserver started at http://127.5.61.62:43611/ using document root <none> and password file <none>
I20260809 20:17:17.846099 7391 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:17.846187 7391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:17.846527 7391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:17.849462 7391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "133721837e8b4b5ead5a8e556fa6e8f4"
format_stamp: "Formatted at 2026-08-09 20:17:17 on dist-test-slave-6k22"
server_key: "47faf332332e3bce689dcfe8741c0dfc"
server_key_iv: "8c6a46430ad5e71fb9c978ac872ae7f8"
server_key_version: "encryptionkey@0"
I20260809 20:17:17.850181 7391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "133721837e8b4b5ead5a8e556fa6e8f4"
format_stamp: "Formatted at 2026-08-09 20:17:17 on dist-test-slave-6k22"
server_key: "47faf332332e3bce689dcfe8741c0dfc"
server_key_iv: "8c6a46430ad5e71fb9c978ac872ae7f8"
server_key_version: "encryptionkey@0"
I20260809 20:17:17.855162 7391 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 20:17:17.858484 7410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:17.859882 7391 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:17:17.860061 7391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "133721837e8b4b5ead5a8e556fa6e8f4"
format_stamp: "Formatted at 2026-08-09 20:17:17 on dist-test-slave-6k22"
server_key: "47faf332332e3bce689dcfe8741c0dfc"
server_key_iv: "8c6a46430ad5e71fb9c978ac872ae7f8"
server_key_version: "encryptionkey@0"
I20260809 20:17:17.860265 7391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:17.890401 7391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:17.894306 7391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:17.894588 7391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:17.915711 7391 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:46769
I20260809 20:17:17.915727 7480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:46769 every 8 connection(s)
I20260809 20:17:17.917039 7391 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:17:17.917950 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 7391
I20260809 20:17:17.918223 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:17:17.918931 5364 external_mini_cluster.cc:1428] Setting key 6dd0d918190411e442b7e5c25e3627d6
I20260809 20:17:17.921597 7481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 20:17:17 dist-test-slave-6k22 krb5kdc[7376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:17.934489 7481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4: Bootstrap starting.
I20260809 20:17:17.937820 7481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:17.938872 7481 log.cc:824] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:17.941677 7481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4: No bootstrap required, opened a new log
I20260809 20:17:17.941931 7484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:17.920884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34094 (local address 127.5.61.62:46769)
0809 20:17:17.921718 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:17.921733 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:17.921782 (+ 49us) server_negotiation.cc:407] Connection header received
0809 20:17:17.924171 (+ 2389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:17.924197 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:17.924686 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:17.925130 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:17.926364 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:17.928440 (+ 2076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:17.929635 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:17.930673 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:17.930858 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:17.935045 (+ 4187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:17.935052 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:17.935065 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:17.935100 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:17.937957 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:17.938650 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:17.938660 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:17.938672 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:17.938842 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:17.939232 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:17.939248 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:17.939255 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:17.939826 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:17.940134 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:17.940578 (+ 444us) server_negotiation.cc:299] Negotiation successful
0809 20:17:17.940966 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":354,"threads_started":1}
I20260809 20:17:17.950455 7481 raft_consensus.cc:348] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "133721837e8b4b5ead5a8e556fa6e8f4" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46769 } }
I20260809 20:17:17.950788 7481 raft_consensus.cc:374] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:17.950909 7481 raft_consensus.cc:729] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 133721837e8b4b5ead5a8e556fa6e8f4, State: Initialized, Role: FOLLOWER
I20260809 20:17:17.951440 7481 consensus_queue.cc:260] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "133721837e8b4b5ead5a8e556fa6e8f4" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46769 } }
I20260809 20:17:17.951699 7481 raft_consensus.cc:388] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:17:17.951848 7481 raft_consensus.cc:482] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:17:17.952039 7481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:17.954067 7481 raft_consensus.cc:504] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "133721837e8b4b5ead5a8e556fa6e8f4" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46769 } }
I20260809 20:17:17.954555 7481 leader_election.cc:302] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 133721837e8b4b5ead5a8e556fa6e8f4; no voters:
I20260809 20:17:17.955200 7481 leader_election.cc:288] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:17:17.955331 7486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:17.956069 7486 raft_consensus.cc:686] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4 [term 1 LEADER]: Becoming Leader. State: Replica: 133721837e8b4b5ead5a8e556fa6e8f4, State: Running, Role: LEADER
I20260809 20:17:17.956601 7486 consensus_queue.cc:237] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "133721837e8b4b5ead5a8e556fa6e8f4" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46769 } }
I20260809 20:17:17.957304 7481 sys_catalog.cc:564] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:17:17.959498 7487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "133721837e8b4b5ead5a8e556fa6e8f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "133721837e8b4b5ead5a8e556fa6e8f4" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46769 } } }
I20260809 20:17:17.959659 7489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 133721837e8b4b5ead5a8e556fa6e8f4. Latest consensus state: current_term: 1 leader_uuid: "133721837e8b4b5ead5a8e556fa6e8f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "133721837e8b4b5ead5a8e556fa6e8f4" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46769 } } }
I20260809 20:17:17.959996 7487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4 [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:17.960033 7489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4 [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:17.962383 7495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:17:17.967820 7495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:17:17.976078 7495 catalog_manager.cc:1347] Generated new cluster ID: 28bb88c4087f4215a1eabec1f8a7e43e
I20260809 20:17:17.976193 7495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:17:17.987360 7495 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:17:17.988566 7495 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:17:17.997553 7495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4: Generated new TSK 0
I20260809 20:17:17.998332 7495 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.61.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
I20260809 20:17:18.051167 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:46769
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:40081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:18.376466 7509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:18.376734 7509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:18.376914 7509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:18.384001 7509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:18.384146 7509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:18.384290 7509 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 20:17:18.384374 7509 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 20:17:18.384452 7509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:17:18.393983 7509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:40081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.61.62:46769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:18.395226 7509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:18.396574 7509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:18.409652 7515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:18.409670 7518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:18.409813 7516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:18.410948 7509 server_base.cc:1034] running on GCE node
I20260809 20:17:18.411634 7509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:18.412595 7509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:18.413841 7509 hybrid_clock.cc:648] HybridClock initialized: now 1786306638413780 us; error 56 us; skew 500 ppm
I20260809 20:17:18.414160 7509 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:18 dist-test-slave-6k22 krb5kdc[7376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306638, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:18.417528 7509 init.cc:377] Logged in from keytab as kudu/127.5.61.1@KRBTEST.COM (short username kudu)
I20260809 20:17:18.418975 7509 webserver.cc:460] Webserver started at http://127.5.61.1:35171/ using document root <none> and password file <none>
I20260809 20:17:18.419701 7509 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:18.419809 7509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:18.420069 7509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:18.422878 7509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "cf9e170e36ea473ab8dedfbf3e237ca5"
format_stamp: "Formatted at 2026-08-09 20:17:18 on dist-test-slave-6k22"
server_key: "be5ce3d10af35c8d1c3281808b0257d8"
server_key_iv: "cdcb373498ca3256b4635c7b131661b0"
server_key_version: "encryptionkey@0"
I20260809 20:17:18.423635 7509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "cf9e170e36ea473ab8dedfbf3e237ca5"
format_stamp: "Formatted at 2026-08-09 20:17:18 on dist-test-slave-6k22"
server_key: "be5ce3d10af35c8d1c3281808b0257d8"
server_key_iv: "cdcb373498ca3256b4635c7b131661b0"
server_key_version: "encryptionkey@0"
I20260809 20:17:18.428521 7509 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 20:17:18.432199 7526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:18.433420 7509 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:17:18.433552 7509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "cf9e170e36ea473ab8dedfbf3e237ca5"
format_stamp: "Formatted at 2026-08-09 20:17:18 on dist-test-slave-6k22"
server_key: "be5ce3d10af35c8d1c3281808b0257d8"
server_key_iv: "cdcb373498ca3256b4635c7b131661b0"
server_key_version: "encryptionkey@0"
I20260809 20:17:18.433714 7509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:18.459103 7509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:18.463696 7509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:18.464026 7509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:18.466739 7509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:18.466833 7509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:18.467000 7509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:18.467098 7509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 20:17:18 dist-test-slave-6k22 krb5kdc[7376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306638, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:18.492937 7538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:18.468437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46769 (local address 127.5.61.1:52339)
0809 20:17:18.469412 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:18.469469 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:18.471388 (+ 1919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:18.472245 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:18.472253 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:18.472998 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:18.473866 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:18.473898 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:18.475597 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:18.475604 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:18.477885 (+ 2281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:18.477898 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:18.478785 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:18.478793 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:18.479165 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:18.480263 (+ 1098us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:18.480317 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:18.484069 (+ 3752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:18.488204 (+ 4135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:18.488213 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:18.488229 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:18.488797 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:18.489213 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:18.489217 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:18.489221 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:18.489459 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:18.490129 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:18.490138 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:18.490563 (+ 425us) client_negotiation.cc:769] Sending connection context
0809 20:17:18.490837 (+ 274us) client_negotiation.cc:240] Negotiation successful
0809 20:17:18.491348 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"thread_start_us":399,"threads_started":1}
I20260809 20:17:18.494408 7536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:18.467519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:52339 (local address 127.5.61.62:46769)
0809 20:17:18.468932 (+ 1413us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:18.468938 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:18.469601 (+ 663us) server_negotiation.cc:407] Connection header received
0809 20:17:18.471744 (+ 2143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:18.471753 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:18.471847 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:18.472001 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:18.474081 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:18.475387 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:18.478066 (+ 2679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:18.478513 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:18.478655 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:18.484382 (+ 5727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:18.484387 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:18.484391 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:18.484425 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:18.488042 (+ 3617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:18.488939 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:18.488944 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:18.488947 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:18.489093 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:18.489584 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:18.489589 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:18.489592 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:18.489929 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:18.490061 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:18.493698 (+ 3637us) server_negotiation.cc:299] Negotiation successful
0809 20:17:18.493954 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1272,"thread_start_us":1071,"threads_started":1}
I20260809 20:17:18.524384 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:18.505511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46769 (local address 127.5.61.1:40835)
0809 20:17:18.506290 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:18.506313 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:18.506531 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:18.507075 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:18.507082 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:18.507778 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:18.508180 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:18.508196 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:18.510061 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:18.510068 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:18.512049 (+ 1981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:18.512063 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:18.513477 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:18.513483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:18.513761 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:18.514770 (+ 1009us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:18.514800 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:18.516195 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:18.520797 (+ 4602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:18.520805 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:18.520812 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:18.521521 (+ 709us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:18.522032 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:18.522038 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:18.522043 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:18.522261 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:18.522986 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:18.522993 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:18.523221 (+ 228us) client_negotiation.cc:769] Sending connection context
0809 20:17:18.523386 (+ 165us) client_negotiation.cc:240] Negotiation successful
0809 20:17:18.523787 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":397,"threads_started":1}
I20260809 20:17:18.524457 7536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:18.505053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:40835 (local address 127.5.61.62:46769)
0809 20:17:18.505271 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:18.505278 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:18.506411 (+ 1133us) server_negotiation.cc:407] Connection header received
0809 20:17:18.506675 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:18.506682 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:18.506765 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:18.506842 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:18.508414 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:18.509857 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:18.512229 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:18.513010 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:18.513132 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:18.516426 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:18.516435 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:18.516444 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:18.516491 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:18.520652 (+ 4161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:18.521705 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:18.521713 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:18.521717 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:18.521892 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:18.522380 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:18.522387 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:18.522394 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:18.522842 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:18.523036 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:18.523604 (+ 568us) server_negotiation.cc:299] Negotiation successful
0809 20:17:18.523969 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 20:17:18.539259 7509 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:43907
I20260809 20:17:18.539342 7671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:43907 every 8 connection(s)
I20260809 20:17:18.540520 7509 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:17:18.542824 7672 heartbeater.cc:347] Connected to a master server at 127.5.61.62:46769
I20260809 20:17:18.543067 7672 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:18.543919 7672 heartbeater.cc:511] Master 127.5.61.62:46769 requested a full tablet report, sending...
I20260809 20:17:18.546036 7429 ts_manager.cc:194] Registered new tserver with Master: cf9e170e36ea473ab8dedfbf3e237ca5 (127.5.61.1:43907)
I20260809 20:17:18.546294 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 7509
I20260809 20:17:18.546501 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:17:18.547353 5364 external_mini_cluster.cc:1428] Setting key 9476c9fb20d976a73618abaaa1287df2
I20260809 20:17:18.547760 7429 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.1@KRBTEST.COM'} at 127.5.61.1:52339
WARNING: no policy specified for kudu/127.5.61.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
I20260809 20:17:18.599509 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:46769
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:40081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:18.936233 7676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:18.936501 7676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:18.936687 7676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:18.944233 7676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:18.944339 7676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:18.944424 7676 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 20:17:18.944478 7676 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 20:17:18.944552 7676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:17:18.954447 7676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:40081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.61.62:46769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:18.955703 7676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:18.956954 7676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:18.970634 7682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:18.970577 7685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:18.970674 7689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:18.971974 7676 server_base.cc:1034] running on GCE node
I20260809 20:17:18.972586 7676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:18.973522 7676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:18.974745 7676 hybrid_clock.cc:648] HybridClock initialized: now 1786306638974706 us; error 47 us; skew 500 ppm
I20260809 20:17:18.975075 7676 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:18 dist-test-slave-6k22 krb5kdc[7376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306638, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:18.978226 7676 init.cc:377] Logged in from keytab as kudu/127.5.61.2@KRBTEST.COM (short username kudu)
I20260809 20:17:18.979810 7676 webserver.cc:460] Webserver started at http://127.5.61.2:35089/ using document root <none> and password file <none>
I20260809 20:17:18.980533 7676 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:18.980679 7676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:18.981014 7676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:18.984122 7676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "a57c0527f7914120af7d94e1be9592c8"
format_stamp: "Formatted at 2026-08-09 20:17:18 on dist-test-slave-6k22"
server_key: "615c085a53725af1f442a0d35f898ff9"
server_key_iv: "b49e18701b3236e66cee1c927f2f366f"
server_key_version: "encryptionkey@0"
I20260809 20:17:18.984990 7676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "a57c0527f7914120af7d94e1be9592c8"
format_stamp: "Formatted at 2026-08-09 20:17:18 on dist-test-slave-6k22"
server_key: "615c085a53725af1f442a0d35f898ff9"
server_key_iv: "b49e18701b3236e66cee1c927f2f366f"
server_key_version: "encryptionkey@0"
I20260809 20:17:18.992551 7676 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260809 20:17:18.997555 7697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:18.999439 7676 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260809 20:17:18.999648 7676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "a57c0527f7914120af7d94e1be9592c8"
format_stamp: "Formatted at 2026-08-09 20:17:18 on dist-test-slave-6k22"
server_key: "615c085a53725af1f442a0d35f898ff9"
server_key_iv: "b49e18701b3236e66cee1c927f2f366f"
server_key_version: "encryptionkey@0"
I20260809 20:17:18.999912 7676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:19.026188 7676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:19.031682 7676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:19.032001 7676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:19.034986 7676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:19.035130 7676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:19.035272 7676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:19.035372 7676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 20:17:19 dist-test-slave-6k22 krb5kdc[7376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306638, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:19.063470 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.036057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:41069 (local address 127.5.61.62:46769)
0809 20:17:19.036519 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:19.036523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:19.038122 (+ 1599us) server_negotiation.cc:407] Connection header received
0809 20:17:19.041822 (+ 3700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:19.041830 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:19.041932 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:19.042634 (+ 702us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:19.043954 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.045198 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.048472 (+ 3274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.049310 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.049426 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.055160 (+ 5734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:19.055167 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:19.055173 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:19.055214 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:19.059486 (+ 4272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:19.060769 (+ 1283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:19.060777 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:19.060783 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:19.060956 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:19.061636 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:19.061643 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:19.061647 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:19.062017 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:19.062173 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:19.062865 (+ 692us) server_negotiation.cc:299] Negotiation successful
0809 20:17:19.063084 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":250,"threads_started":1}
I20260809 20:17:19.064567 7709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.036710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46769 (local address 127.5.61.2:41069)
0809 20:17:19.037720 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:19.037774 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:19.041471 (+ 3697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:19.042431 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:19.042443 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:19.043138 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:19.043767 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:19.043786 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.045419 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.045428 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:19.048258 (+ 2830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:19.048275 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.049935 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.049947 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:19.050324 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.051266 (+ 942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:19.051306 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:19.054911 (+ 3605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:19.059713 (+ 4802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:19.059726 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:19.059751 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:19.060552 (+ 801us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:19.061094 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:19.061100 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:19.061105 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:19.061451 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:19.062168 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:19.062175 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:19.062492 (+ 317us) client_negotiation.cc:769] Sending connection context
0809 20:17:19.062756 (+ 264us) client_negotiation.cc:240] Negotiation successful
0809 20:17:19.063310 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":409,"threads_started":1}
I20260809 20:17:19.081470 7676 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:36835
I20260809 20:17:19.081525 7836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:36835 every 8 connection(s)
I20260809 20:17:19.082855 7676 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:17:19.085165 7837 heartbeater.cc:347] Connected to a master server at 127.5.61.62:46769
I20260809 20:17:19.085414 7837 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:19.086097 7837 heartbeater.cc:511] Master 127.5.61.62:46769 requested a full tablet report, sending...
I20260809 20:17:19.086702 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 7676
I20260809 20:17:19.086920 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:17:19.087527 5364 external_mini_cluster.cc:1428] Setting key 4b762270795870dbde688af975a3a5d3
I20260809 20:17:19.087529 7428 ts_manager.cc:194] Registered new tserver with Master: a57c0527f7914120af7d94e1be9592c8 (127.5.61.2:36835)
I20260809 20:17:19.088032 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.072512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:37755 (local address 127.5.61.62:46769)
0809 20:17:19.073032 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:19.073039 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:19.073524 (+ 485us) server_negotiation.cc:407] Connection header received
0809 20:17:19.073734 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:19.073768 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:19.073862 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:19.073980 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:19.075531 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.076679 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.078793 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.079244 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.079381 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.081749 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:19.081754 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:19.081760 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:19.081796 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:19.084808 (+ 3012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:19.085677 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:19.085683 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:19.085687 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:19.085818 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:19.086218 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:19.086223 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:19.086226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:19.086692 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:19.086867 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:19.087443 (+ 576us) server_negotiation.cc:299] Negotiation successful
0809 20:17:19.087726 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384}
I20260809 20:17:19.088083 7810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.072559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46769 (local address 127.5.61.2:37755)
0809 20:17:19.073422 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:19.073443 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:19.073595 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:19.074170 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:19.074173 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:19.074842 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:19.075302 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:19.075321 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.076834 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.076840 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:19.078627 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:19.078639 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.079450 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.079457 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:19.079664 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.080486 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:19.080506 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:19.081485 (+ 979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:19.084944 (+ 3459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:19.084949 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:19.084953 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:19.085522 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:19.085931 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:19.085936 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:19.085940 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:19.086109 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:19.086938 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:19.086944 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:19.087141 (+ 197us) client_negotiation.cc:769] Sending connection context
0809 20:17:19.087311 (+ 170us) client_negotiation.cc:240] Negotiation successful
0809 20:17:19.087661 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"thread_start_us":508,"threads_started":1}
I20260809 20:17:19.088771 7428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.2@KRBTEST.COM'} at 127.5.61.2:41069
WARNING: no policy specified for kudu/127.5.61.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
I20260809 20:17:19.132225 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:46769
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:40081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:19.457046 7842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:19.457407 7842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:19.457595 7842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:19.465073 7842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:19.465219 7842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:19.465365 7842 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 20:17:19.465451 7842 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 20:17:19.465584 7842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:17:19.475322 7842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:40081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.61.62:46769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:19.476610 7842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:19.477962 7842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:19.491001 7849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:19.490967 7848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:19.491036 7852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:19.491881 7842 server_base.cc:1034] running on GCE node
I20260809 20:17:19.492517 7842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:19.493429 7842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:19.494683 7842 hybrid_clock.cc:648] HybridClock initialized: now 1786306639494624 us; error 38 us; skew 500 ppm
I20260809 20:17:19.495012 7842 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:19 dist-test-slave-6k22 krb5kdc[7376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306639, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:19.498345 7842 init.cc:377] Logged in from keytab as kudu/127.5.61.3@KRBTEST.COM (short username kudu)
I20260809 20:17:19.499801 7842 webserver.cc:460] Webserver started at http://127.5.61.3:40637/ using document root <none> and password file <none>
I20260809 20:17:19.500465 7842 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:19.500612 7842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:19.500926 7842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:19.503890 7842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "b02be2e464f84631af2c4dc241d307d5"
format_stamp: "Formatted at 2026-08-09 20:17:19 on dist-test-slave-6k22"
server_key: "ce81d7ff6f3131ead7bd8ced90a520ed"
server_key_iv: "87f109bc92b7bc20794ecc5cd20d3326"
server_key_version: "encryptionkey@0"
I20260809 20:17:19.504659 7842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "b02be2e464f84631af2c4dc241d307d5"
format_stamp: "Formatted at 2026-08-09 20:17:19 on dist-test-slave-6k22"
server_key: "ce81d7ff6f3131ead7bd8ced90a520ed"
server_key_iv: "87f109bc92b7bc20794ecc5cd20d3326"
server_key_version: "encryptionkey@0"
I20260809 20:17:19.509681 7842 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 20:17:19.513186 7861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:19.514865 7842 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 20:17:19.515039 7842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "b02be2e464f84631af2c4dc241d307d5"
format_stamp: "Formatted at 2026-08-09 20:17:19 on dist-test-slave-6k22"
server_key: "ce81d7ff6f3131ead7bd8ced90a520ed"
server_key_iv: "87f109bc92b7bc20794ecc5cd20d3326"
server_key_version: "encryptionkey@0"
I20260809 20:17:19.515321 7842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:19.532222 7842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:19.536048 7842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:19.536329 7842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:19.539206 7842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:19.539323 7842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:19.539458 7842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:19.539577 7842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:19.551591 7672 heartbeater.cc:503] Master 127.5.61.62:46769 was elected leader, sending a full tablet report...
Aug 09 20:17:19 dist-test-slave-6k22 krb5kdc[7376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306639, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:19.566802 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.539745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:37137 (local address 127.5.61.62:46769)
0809 20:17:19.540419 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:19.540426 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:19.541458 (+ 1032us) server_negotiation.cc:407] Connection header received
0809 20:17:19.544658 (+ 3200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:19.544665 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:19.544749 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:19.544822 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:19.547290 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.548644 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.551272 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.552139 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.552290 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.558037 (+ 5747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:19.558046 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:19.558053 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:19.558109 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:19.562280 (+ 4171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:19.563592 (+ 1312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:19.563621 (+ 29us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:19.563628 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:19.563817 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:19.564539 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:19.564543 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:19.564547 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:19.564940 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:19.565102 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:19.566139 (+ 1037us) server_negotiation.cc:299] Negotiation successful
0809 20:17:19.566382 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523}
I20260809 20:17:19.568459 7871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.540367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46769 (local address 127.5.61.3:37137)
0809 20:17:19.541290 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:19.541336 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:19.544316 (+ 2980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:19.545353 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:19.545368 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:19.546206 (+ 838us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:19.547067 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:19.547107 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.548862 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.548869 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:19.551088 (+ 2219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:19.551108 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.552702 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.552713 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:19.553116 (+ 403us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.553987 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:19.554024 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:19.557744 (+ 3720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:19.562460 (+ 4716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:19.562477 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:19.562498 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:19.563350 (+ 852us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:19.564005 (+ 655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:19.564016 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:19.564023 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:19.564382 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:19.565143 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:19.565158 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:19.565638 (+ 480us) client_negotiation.cc:769] Sending connection context
0809 20:17:19.566030 (+ 392us) client_negotiation.cc:240] Negotiation successful
0809 20:17:19.566623 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":414,"threads_started":1}
I20260809 20:17:19.588342 7842 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:38807
I20260809 20:17:19.588434 8002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:38807 every 8 connection(s)
I20260809 20:17:19.589502 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.575205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:50047 (local address 127.5.61.62:46769)
0809 20:17:19.575380 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:19.575386 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:19.576313 (+ 927us) server_negotiation.cc:407] Connection header received
0809 20:17:19.576538 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:19.576543 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:19.576628 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:19.576735 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:19.577584 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.578289 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.579724 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.580642 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.580812 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.583414 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:19.583420 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:19.583425 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:19.583464 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:19.586591 (+ 3127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:19.587319 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:19.587326 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:19.587330 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:19.587462 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:19.587883 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:19.587890 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:19.587894 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:19.588256 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:19.588406 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:19.588931 (+ 525us) server_negotiation.cc:299] Negotiation successful
0809 20:17:19.589134 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 20:17:19.589592 7977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.575350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46769 (local address 127.5.61.3:50047)
0809 20:17:19.576177 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:19.576203 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:19.576370 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:19.576758 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:19.576763 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:19.577173 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:19.577448 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:19.577456 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.578464 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.578470 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:19.579503 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:19.579521 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.580835 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.580839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:19.580961 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.581682 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:19.581725 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:19.583221 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:19.586728 (+ 3507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:19.586733 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:19.586736 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:19.587192 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:19.587608 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:19.587612 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:19.587619 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:19.587754 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:19.588477 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:19.588482 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:19.588673 (+ 191us) client_negotiation.cc:769] Sending connection context
0809 20:17:19.588819 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 20:17:19.589128 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":463,"threads_started":1}
I20260809 20:17:19.589699 7842 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:17:19.590247 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 7842
I20260809 20:17:19.590426 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:17:19.590875 5364 external_mini_cluster.cc:1428] Setting key e4abfdd5451b1bc0fd97a6c7ba8f0ac7
I20260809 20:17:19.594899 8004 heartbeater.cc:347] Connected to a master server at 127.5.61.62:46769
I20260809 20:17:19.595443 8004 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:19.596112 8004 heartbeater.cc:511] Master 127.5.61.62:46769 requested a full tablet report, sending...
I20260809 20:17:19.597457 7427 ts_manager.cc:194] Registered new tserver with Master: b02be2e464f84631af2c4dc241d307d5 (127.5.61.3:38807)
I20260809 20:17:19.598390 7427 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.3@KRBTEST.COM'} at 127.5.61.3:37137
I20260809 20:17:19.606520 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 20:17:19 dist-test-slave-6k22 krb5kdc[7376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 20:17:19 dist-test-slave-6k22 krb5kdc[7376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:19.655668 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.641587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55736 (local address 127.5.61.62:46769)
0809 20:17:19.641833 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:19.641842 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:19.642133 (+ 291us) server_negotiation.cc:407] Connection header received
0809 20:17:19.642358 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:19.642370 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:19.642447 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:19.642555 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:19.644043 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.644812 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.646149 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.646568 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.646710 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.650585 (+ 3875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:19.650590 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:19.650594 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:19.650627 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:19.653427 (+ 2800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:19.654007 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:19.654010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:19.654013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:19.654135 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:19.654463 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:19.654466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:19.654468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:19.654726 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:19.654916 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:19.655173 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 20:17:19.655351 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 20:17:19.659698 7427 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55736:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:17:19.666719 7427 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:17:19.701771 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.678484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53930 (local address 127.5.61.3:38807)
0809 20:17:19.691192 (+ 12708us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:19.691203 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:19.691245 (+ 42us) server_negotiation.cc:407] Connection header received
0809 20:17:19.691393 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:19.691404 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:19.691725 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:19.691941 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:19.693108 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.695444 (+ 2336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.699040 (+ 3596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.700496 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.700636 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.700910 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:19.701245 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 20:17:19.701414 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12492,"thread_start_us":12626,"threads_started":1}
I20260809 20:17:19.701932 8018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.678153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:38807 (local address 127.0.0.1:53930)
0809 20:17:19.690772 (+ 12619us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:19.690806 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:19.690972 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:19.692278 (+ 1306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:19.692287 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:19.692325 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:19.692841 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:19.692859 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.695667 (+ 2808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.695675 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:19.698878 (+ 3203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:19.698890 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.700706 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.700711 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:19.700951 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.701031 (+ 80us) client_negotiation.cc:769] Sending connection context
0809 20:17:19.701177 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 20:17:19.701471 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12339,"mutex_wait_us":97,"thread_start_us":12179,"threads_started":1}
I20260809 20:17:19.702099 8022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.678904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47300 (local address 127.5.61.1:43907)
0809 20:17:19.691960 (+ 13056us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:19.691965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:19.691989 (+ 24us) server_negotiation.cc:407] Connection header received
0809 20:17:19.692059 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:19.692064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:19.692215 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:19.692413 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:19.693476 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.695427 (+ 1951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.699111 (+ 3684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.700530 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.700689 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.700913 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:19.701597 (+ 684us) server_negotiation.cc:299] Negotiation successful
0809 20:17:19.701775 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12897,"thread_start_us":12684,"threads_started":1}
I20260809 20:17:19.702770 8019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.678254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:43907 (local address 127.0.0.1:47300)
0809 20:17:19.691818 (+ 13564us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:19.691838 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:19.691956 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:19.692630 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:19.692635 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:19.692667 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:19.693227 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:19.693252 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.695704 (+ 2452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.695713 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:19.698942 (+ 3229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:19.698954 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.701095 (+ 2141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.701100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:19.701291 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.701418 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 20:17:19.702366 (+ 948us) client_negotiation.cc:240] Negotiation successful
0809 20:17:19.702477 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13446,"mutex_wait_us":164,"thread_start_us":13781,"threads_started":1}
I20260809 20:17:19.705710 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.692407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:36835 (local address 127.0.0.1:57324)
0809 20:17:19.697623 (+ 5216us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:19.697642 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:19.697788 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:19.698365 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:19.698381 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:19.698406 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:19.698887 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:19.698900 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.700784 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.700789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:19.703134 (+ 2345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:19.703149 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.704906 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.704912 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:19.705085 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.705117 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 20:17:19.705281 (+ 164us) client_negotiation.cc:240] Negotiation successful
0809 20:17:19.705411 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5080,"thread_start_us":4901,"threads_started":1}
I20260809 20:17:19.705710 8024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.692660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57324 (local address 127.5.61.2:36835)
0809 20:17:19.696268 (+ 3608us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:19.696276 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:19.697765 (+ 1489us) server_negotiation.cc:407] Connection header received
0809 20:17:19.697939 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:19.697946 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:19.698167 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:19.698349 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:19.699141 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.700625 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.703334 (+ 2709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.704699 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.704833 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.705027 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:19.705263 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 20:17:19.705431 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3452,"thread_start_us":3653,"threads_started":1}
I20260809 20:17:19.706295 7919 tablet_service.cc:1467] Processing CreateTablet for tablet 09ad6ef7fcdb414ca34369f9db54234e (DEFAULT_TABLE table=test-table [id=37cfd41ef22442998bc22259883d76de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:19.706336 7591 tablet_service.cc:1467] Processing CreateTablet for tablet 09ad6ef7fcdb414ca34369f9db54234e (DEFAULT_TABLE table=test-table [id=37cfd41ef22442998bc22259883d76de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:19.707729 7919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09ad6ef7fcdb414ca34369f9db54234e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:19.708767 7591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09ad6ef7fcdb414ca34369f9db54234e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:19.709251 7756 tablet_service.cc:1467] Processing CreateTablet for tablet 09ad6ef7fcdb414ca34369f9db54234e (DEFAULT_TABLE table=test-table [id=37cfd41ef22442998bc22259883d76de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:19.710552 7756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09ad6ef7fcdb414ca34369f9db54234e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:19.719062 8025 tablet_bootstrap.cc:492] T 09ad6ef7fcdb414ca34369f9db54234e P b02be2e464f84631af2c4dc241d307d5: Bootstrap starting.
I20260809 20:17:19.722620 8025 tablet_bootstrap.cc:654] T 09ad6ef7fcdb414ca34369f9db54234e P b02be2e464f84631af2c4dc241d307d5: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:19.723438 8026 tablet_bootstrap.cc:492] T 09ad6ef7fcdb414ca34369f9db54234e P cf9e170e36ea473ab8dedfbf3e237ca5: Bootstrap starting.
I20260809 20:17:19.724123 8025 log.cc:824] T 09ad6ef7fcdb414ca34369f9db54234e P b02be2e464f84631af2c4dc241d307d5: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:19.724972 8027 tablet_bootstrap.cc:492] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8: Bootstrap starting.
I20260809 20:17:19.727674 8026 tablet_bootstrap.cc:654] T 09ad6ef7fcdb414ca34369f9db54234e P cf9e170e36ea473ab8dedfbf3e237ca5: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:19.727708 8027 tablet_bootstrap.cc:654] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:19.728765 8026 log.cc:824] T 09ad6ef7fcdb414ca34369f9db54234e P cf9e170e36ea473ab8dedfbf3e237ca5: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:19.728948 8027 log.cc:824] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:19.730181 8025 tablet_bootstrap.cc:492] T 09ad6ef7fcdb414ca34369f9db54234e P b02be2e464f84631af2c4dc241d307d5: No bootstrap required, opened a new log
I20260809 20:17:19.730495 8025 ts_tablet_manager.cc:1397] T 09ad6ef7fcdb414ca34369f9db54234e P b02be2e464f84631af2c4dc241d307d5: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260809 20:17:19.732513 8026 tablet_bootstrap.cc:492] T 09ad6ef7fcdb414ca34369f9db54234e P cf9e170e36ea473ab8dedfbf3e237ca5: No bootstrap required, opened a new log
I20260809 20:17:19.732762 8026 ts_tablet_manager.cc:1397] T 09ad6ef7fcdb414ca34369f9db54234e P cf9e170e36ea473ab8dedfbf3e237ca5: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260809 20:17:19.739353 8027 tablet_bootstrap.cc:492] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8: No bootstrap required, opened a new log
I20260809 20:17:19.739671 8027 ts_tablet_manager.cc:1397] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.008s
I20260809 20:17:19.741205 8025 raft_consensus.cc:348] T 09ad6ef7fcdb414ca34369f9db54234e P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } } peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } }
I20260809 20:17:19.741639 8025 raft_consensus.cc:374] T 09ad6ef7fcdb414ca34369f9db54234e P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:19.741732 8025 raft_consensus.cc:729] T 09ad6ef7fcdb414ca34369f9db54234e P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b02be2e464f84631af2c4dc241d307d5, State: Initialized, Role: FOLLOWER
I20260809 20:17:19.742354 8025 consensus_queue.cc:260] T 09ad6ef7fcdb414ca34369f9db54234e P b02be2e464f84631af2c4dc241d307d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } } peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } }
I20260809 20:17:19.743482 8004 heartbeater.cc:503] Master 127.5.61.62:46769 was elected leader, sending a full tablet report...
I20260809 20:17:19.743791 8026 raft_consensus.cc:348] T 09ad6ef7fcdb414ca34369f9db54234e P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } } peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } }
I20260809 20:17:19.744235 8025 ts_tablet_manager.cc:1428] T 09ad6ef7fcdb414ca34369f9db54234e P b02be2e464f84631af2c4dc241d307d5: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260809 20:17:19.744234 8026 raft_consensus.cc:374] T 09ad6ef7fcdb414ca34369f9db54234e P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:19.744328 8026 raft_consensus.cc:729] T 09ad6ef7fcdb414ca34369f9db54234e P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf9e170e36ea473ab8dedfbf3e237ca5, State: Initialized, Role: FOLLOWER
I20260809 20:17:19.744963 8026 consensus_queue.cc:260] T 09ad6ef7fcdb414ca34369f9db54234e P cf9e170e36ea473ab8dedfbf3e237ca5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } } peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } }
I20260809 20:17:19.746634 8026 ts_tablet_manager.cc:1428] T 09ad6ef7fcdb414ca34369f9db54234e P cf9e170e36ea473ab8dedfbf3e237ca5: Time spent starting tablet: real 0.014s user 0.007s sys 0.006s
I20260809 20:17:19.750058 8027 raft_consensus.cc:348] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } } peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } }
I20260809 20:17:19.750469 8027 raft_consensus.cc:374] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:19.750583 8027 raft_consensus.cc:729] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a57c0527f7914120af7d94e1be9592c8, State: Initialized, Role: FOLLOWER
I20260809 20:17:19.751050 8027 consensus_queue.cc:260] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } } peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } }
I20260809 20:17:19.752038 7837 heartbeater.cc:503] Master 127.5.61.62:46769 was elected leader, sending a full tablet report...
I20260809 20:17:19.752422 8027 ts_tablet_manager.cc:1428] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
W20260809 20:17:19.793041 7673 tablet.cc:2369] T 09ad6ef7fcdb414ca34369f9db54234e P cf9e170e36ea473ab8dedfbf3e237ca5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:17:19.797214 8034 raft_consensus.cc:482] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:17:19.797533 8034 raft_consensus.cc:504] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } } peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } }
I20260809 20:17:19.799228 8034 leader_election.cc:288] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b02be2e464f84631af2c4dc241d307d5 (127.5.61.3:38807), cf9e170e36ea473ab8dedfbf3e237ca5 (127.5.61.1:43907)
I20260809 20:17:19.806416 8022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.799711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:44619 (local address 127.5.61.1:43907)
0809 20:17:19.799962 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:19.799970 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:19.800605 (+ 635us) server_negotiation.cc:407] Connection header received
0809 20:17:19.800877 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:19.800885 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:19.800985 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:19.801057 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:19.801785 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.802560 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.804620 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.805365 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.805478 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.805556 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:19.806016 (+ 460us) server_negotiation.cc:299] Negotiation successful
0809 20:17:19.806157 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 20:17:19.806499 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.799530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:36927 (local address 127.5.61.3:38807)
0809 20:17:19.799863 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:19.799870 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:19.800778 (+ 908us) server_negotiation.cc:407] Connection header received
0809 20:17:19.800954 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:19.800957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:19.801016 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:19.801137 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:19.801891 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.802580 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.804572 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.805542 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.805691 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.805813 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:19.806120 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 20:17:19.806257 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260809 20:17:19.806587 8038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.799529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:38807 (local address 127.5.61.2:36927)
0809 20:17:19.800669 (+ 1140us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:19.800684 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:19.800826 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:19.801174 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:19.801179 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:19.801207 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:19.801682 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:19.801728 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.802749 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.802753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:19.804412 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:19.804427 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.805795 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.805799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:19.805924 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.805978 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 20:17:19.806076 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 20:17:19.806193 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1022,"mutex_wait_us":4,"thread_start_us":761,"threads_started":1}
I20260809 20:17:19.807019 8035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.799517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:43907 (local address 127.5.61.2:44619)
0809 20:17:19.800492 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:19.800511 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:19.800684 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:19.801122 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:19.801126 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:19.801160 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:19.801603 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:19.801613 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.802700 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.802706 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:19.804436 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:19.804449 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.805620 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.805629 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:19.805848 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.805882 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 20:17:19.805959 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 20:17:19.806069 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":860,"mutex_wait_us":105,"thread_start_us":505,"threads_started":1}
I20260809 20:17:19.807636 7943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09ad6ef7fcdb414ca34369f9db54234e" candidate_uuid: "a57c0527f7914120af7d94e1be9592c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b02be2e464f84631af2c4dc241d307d5" is_pre_election: true
I20260809 20:17:19.807958 7614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09ad6ef7fcdb414ca34369f9db54234e" candidate_uuid: "a57c0527f7914120af7d94e1be9592c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" is_pre_election: true
I20260809 20:17:19.808130 7943 raft_consensus.cc:2454] T 09ad6ef7fcdb414ca34369f9db54234e P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a57c0527f7914120af7d94e1be9592c8 in term 0.
I20260809 20:17:19.808426 7614 raft_consensus.cc:2454] T 09ad6ef7fcdb414ca34369f9db54234e P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a57c0527f7914120af7d94e1be9592c8 in term 0.
I20260809 20:17:19.808804 7699 leader_election.cc:302] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a57c0527f7914120af7d94e1be9592c8, b02be2e464f84631af2c4dc241d307d5; no voters:
I20260809 20:17:19.809262 8034 raft_consensus.cc:2781] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:17:19.809410 8034 raft_consensus.cc:482] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:17:19.809514 8034 raft_consensus.cc:3037] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:19.811826 8034 raft_consensus.cc:504] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } } peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } }
I20260809 20:17:19.812413 8034 leader_election.cc:288] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8 [CANDIDATE]: Term 1 election: Requested vote from peers b02be2e464f84631af2c4dc241d307d5 (127.5.61.3:38807), cf9e170e36ea473ab8dedfbf3e237ca5 (127.5.61.1:43907)
I20260809 20:17:19.812844 7943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09ad6ef7fcdb414ca34369f9db54234e" candidate_uuid: "a57c0527f7914120af7d94e1be9592c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b02be2e464f84631af2c4dc241d307d5"
I20260809 20:17:19.812922 7614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09ad6ef7fcdb414ca34369f9db54234e" candidate_uuid: "a57c0527f7914120af7d94e1be9592c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5"
I20260809 20:17:19.813060 7943 raft_consensus.cc:3037] T 09ad6ef7fcdb414ca34369f9db54234e P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:19.813135 7614 raft_consensus.cc:3037] T 09ad6ef7fcdb414ca34369f9db54234e P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:19.815516 7943 raft_consensus.cc:2454] T 09ad6ef7fcdb414ca34369f9db54234e P b02be2e464f84631af2c4dc241d307d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a57c0527f7914120af7d94e1be9592c8 in term 1.
I20260809 20:17:19.815537 7614 raft_consensus.cc:2454] T 09ad6ef7fcdb414ca34369f9db54234e P cf9e170e36ea473ab8dedfbf3e237ca5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a57c0527f7914120af7d94e1be9592c8 in term 1.
I20260809 20:17:19.815910 7699 leader_election.cc:302] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a57c0527f7914120af7d94e1be9592c8, b02be2e464f84631af2c4dc241d307d5; no voters:
I20260809 20:17:19.816211 8034 raft_consensus.cc:2781] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:19.816670 8034 raft_consensus.cc:686] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8 [term 1 LEADER]: Becoming Leader. State: Replica: a57c0527f7914120af7d94e1be9592c8, State: Running, Role: LEADER
I20260809 20:17:19.817121 8034 consensus_queue.cc:237] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } } peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } }
I20260809 20:17:19.821841 7427 catalog_manager.cc:5495] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a57c0527f7914120af7d94e1be9592c8 (127.5.61.2). New cstate: current_term: 1 leader_uuid: "a57c0527f7914120af7d94e1be9592c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } health_report { overall_health: HEALTHY } } }
W20260809 20:17:19.835368 7838 tablet.cc:2369] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 20:17:19.845463 8005 tablet.cc:2369] T 09ad6ef7fcdb414ca34369f9db54234e P b02be2e464f84631af2c4dc241d307d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 20:17:19 dist-test-slave-6k22 krb5kdc[7376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306637, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:19.890877 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.877780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55752 (local address 127.5.61.62:46769)
0809 20:17:19.877948 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:19.877952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:19.878484 (+ 532us) server_negotiation.cc:407] Connection header received
0809 20:17:19.878690 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:19.878695 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:19.878763 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:19.878862 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:19.880085 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.880780 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.882045 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.882459 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.882596 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.885834 (+ 3238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:19.885838 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:19.885842 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:19.885868 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:19.888187 (+ 2319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:19.888843 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:19.888850 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:19.888856 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:19.888978 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:19.889341 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:19.889348 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:19.889351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:19.889618 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:19.889800 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:19.890179 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 20:17:19.890487 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":123,"server-negotiator.queue_time_us":56}
I20260809 20:17:19.890909 8048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.877783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46769 (local address 127.0.0.1:55752)
0809 20:17:19.878360 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:19.878391 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:19.878564 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:19.878890 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:19.878895 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:19.879656 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:19.879949 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:19.879960 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.880921 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.880929 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:19.881905 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:19.881917 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.882638 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.882645 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:19.882837 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.883445 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:19.883466 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:19.885640 (+ 2174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:19.888318 (+ 2678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:19.888330 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:19.888338 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:19.888728 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:19.889081 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:19.889087 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:19.889091 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:19.889232 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:19.889739 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:19.889746 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:19.889932 (+ 186us) client_negotiation.cc:769] Sending connection context
0809 20:17:19.890090 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 20:17:19.890369 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":302,"threads_started":1}
I20260809 20:17:19.895766 7427 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:55752:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 20:17:19.896822 7427 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:17:19.903175 7591 tablet_service.cc:1467] Processing CreateTablet for tablet c65b548fff22408aaa1b7147f6a1d07f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=81bf6af243c2452190d33cf6fd7a1819]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 20:17:19.903208 7756 tablet_service.cc:1467] Processing CreateTablet for tablet c65b548fff22408aaa1b7147f6a1d07f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=81bf6af243c2452190d33cf6fd7a1819]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 20:17:19.903427 7919 tablet_service.cc:1467] Processing CreateTablet for tablet c65b548fff22408aaa1b7147f6a1d07f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=81bf6af243c2452190d33cf6fd7a1819]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 20:17:19.904211 7591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c65b548fff22408aaa1b7147f6a1d07f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:19.904232 7756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c65b548fff22408aaa1b7147f6a1d07f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:19.904241 7919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c65b548fff22408aaa1b7147f6a1d07f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:19.910544 8026 tablet_bootstrap.cc:492] T c65b548fff22408aaa1b7147f6a1d07f P cf9e170e36ea473ab8dedfbf3e237ca5: Bootstrap starting.
I20260809 20:17:19.911473 8025 tablet_bootstrap.cc:492] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5: Bootstrap starting.
I20260809 20:17:19.912205 8026 tablet_bootstrap.cc:654] T c65b548fff22408aaa1b7147f6a1d07f P cf9e170e36ea473ab8dedfbf3e237ca5: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:19.913328 8025 tablet_bootstrap.cc:654] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:19.913640 8027 tablet_bootstrap.cc:492] T c65b548fff22408aaa1b7147f6a1d07f P a57c0527f7914120af7d94e1be9592c8: Bootstrap starting.
I20260809 20:17:19.914506 8026 tablet_bootstrap.cc:492] T c65b548fff22408aaa1b7147f6a1d07f P cf9e170e36ea473ab8dedfbf3e237ca5: No bootstrap required, opened a new log
I20260809 20:17:19.914670 8026 ts_tablet_manager.cc:1397] T c65b548fff22408aaa1b7147f6a1d07f P cf9e170e36ea473ab8dedfbf3e237ca5: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 20:17:19.915383 8026 raft_consensus.cc:348] T c65b548fff22408aaa1b7147f6a1d07f P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:19.915660 8026 raft_consensus.cc:374] T c65b548fff22408aaa1b7147f6a1d07f P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:19.915756 8026 raft_consensus.cc:729] T c65b548fff22408aaa1b7147f6a1d07f P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf9e170e36ea473ab8dedfbf3e237ca5, State: Initialized, Role: FOLLOWER
I20260809 20:17:19.915867 8027 tablet_bootstrap.cc:654] T c65b548fff22408aaa1b7147f6a1d07f P a57c0527f7914120af7d94e1be9592c8: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:19.916054 8026 consensus_queue.cc:260] T c65b548fff22408aaa1b7147f6a1d07f P cf9e170e36ea473ab8dedfbf3e237ca5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:19.916548 8026 ts_tablet_manager.cc:1428] T c65b548fff22408aaa1b7147f6a1d07f P cf9e170e36ea473ab8dedfbf3e237ca5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 20:17:19.919801 8033 tablet_replica.cc:439] T c65b548fff22408aaa1b7147f6a1d07f P cf9e170e36ea473ab8dedfbf3e237ca5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } } }
I20260809 20:17:19.920089 8033 tablet_replica.cc:442] T c65b548fff22408aaa1b7147f6a1d07f P cf9e170e36ea473ab8dedfbf3e237ca5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 20:17:19.923377 8027 tablet_bootstrap.cc:492] T c65b548fff22408aaa1b7147f6a1d07f P a57c0527f7914120af7d94e1be9592c8: No bootstrap required, opened a new log
I20260809 20:17:19.923533 8027 ts_tablet_manager.cc:1397] T c65b548fff22408aaa1b7147f6a1d07f P a57c0527f7914120af7d94e1be9592c8: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 20:17:19.924422 8025 tablet_bootstrap.cc:492] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5: No bootstrap required, opened a new log
I20260809 20:17:19.924569 8025 ts_tablet_manager.cc:1397] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.004s
I20260809 20:17:19.924376 8027 raft_consensus.cc:348] T c65b548fff22408aaa1b7147f6a1d07f P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:19.924669 8027 raft_consensus.cc:374] T c65b548fff22408aaa1b7147f6a1d07f P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:19.924780 8027 raft_consensus.cc:729] T c65b548fff22408aaa1b7147f6a1d07f P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a57c0527f7914120af7d94e1be9592c8, State: Initialized, Role: FOLLOWER
I20260809 20:17:19.925007 8027 consensus_queue.cc:260] T c65b548fff22408aaa1b7147f6a1d07f P a57c0527f7914120af7d94e1be9592c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:19.925345 8025 raft_consensus.cc:348] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:19.925607 8025 raft_consensus.cc:374] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:19.925629 8027 ts_tablet_manager.cc:1428] T c65b548fff22408aaa1b7147f6a1d07f P a57c0527f7914120af7d94e1be9592c8: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 20:17:19.925714 8025 raft_consensus.cc:729] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b02be2e464f84631af2c4dc241d307d5, State: Initialized, Role: FOLLOWER
I20260809 20:17:19.925961 8034 tablet_replica.cc:439] T c65b548fff22408aaa1b7147f6a1d07f P a57c0527f7914120af7d94e1be9592c8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } } }
I20260809 20:17:19.926033 8025 consensus_queue.cc:260] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:19.926234 8034 tablet_replica.cc:442] T c65b548fff22408aaa1b7147f6a1d07f P a57c0527f7914120af7d94e1be9592c8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 20:17:19.926545 8031 tablet_replica.cc:439] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } } }
I20260809 20:17:19.926893 8031 tablet_replica.cc:442] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 20:17:19.927702 8025 ts_tablet_manager.cc:1428] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 20:17:19.978461 8031 raft_consensus.cc:482] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:17:19.978752 8031 raft_consensus.cc:504] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:19.980398 8031 leader_election.cc:288] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf9e170e36ea473ab8dedfbf3e237ca5 (127.5.61.1:43907), a57c0527f7914120af7d94e1be9592c8 (127.5.61.2:36835)
I20260809 20:17:19.987098 7871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.980583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:43907 (local address 127.5.61.3:58323)
0809 20:17:19.980776 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:19.980797 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:19.980921 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:19.981450 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:19.981455 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:19.981489 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:19.981708 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:19.981724 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.983131 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.983134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:19.985101 (+ 1967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:19.985117 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.986521 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.986525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:19.986652 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.986675 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 20:17:19.986753 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 20:17:19.986842 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 20:17:19.987227 8022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.980732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:58323 (local address 127.5.61.1:43907)
0809 20:17:19.980927 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:19.980932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:19.980952 (+ 20us) server_negotiation.cc:407] Connection header received
0809 20:17:19.981029 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:19.981035 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:19.981104 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:19.981224 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:19.981921 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.982974 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.985279 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.986379 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.986508 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.986610 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:19.986808 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 20:17:19.986948 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 20:17:19.987926 8024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.981163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:52625 (local address 127.5.61.2:36835)
0809 20:17:19.981406 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:19.981411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:19.981776 (+ 365us) server_negotiation.cc:407] Connection header received
0809 20:17:19.981937 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:19.981942 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:19.982023 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:19.982125 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:19.982565 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.983462 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.985674 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.986887 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.987023 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.987121 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:19.987524 (+ 403us) server_negotiation.cc:299] Negotiation successful
0809 20:17:19.987667 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 20:17:19.987944 8053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:19.980961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:36835 (local address 127.5.61.3:52625)
0809 20:17:19.981675 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:19.981692 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:19.981814 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:19.982136 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:19.982139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:19.982157 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:19.982423 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:19.982430 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.983630 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.983635 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:19.985530 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:19.985540 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:19.987131 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:19.987136 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:19.987327 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:19.987369 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 20:17:19.987495 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 20:17:19.987629 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":353,"threads_started":1}
I20260809 20:17:19.987833 7614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c65b548fff22408aaa1b7147f6a1d07f" candidate_uuid: "b02be2e464f84631af2c4dc241d307d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" is_pre_election: true
I20260809 20:17:19.988116 7614 raft_consensus.cc:2454] T c65b548fff22408aaa1b7147f6a1d07f P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b02be2e464f84631af2c4dc241d307d5 in term 0.
I20260809 20:17:19.988647 7866 leader_election.cc:302] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b02be2e464f84631af2c4dc241d307d5, cf9e170e36ea473ab8dedfbf3e237ca5; no voters:
I20260809 20:17:19.989073 8031 raft_consensus.cc:2781] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:17:19.988941 7780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c65b548fff22408aaa1b7147f6a1d07f" candidate_uuid: "b02be2e464f84631af2c4dc241d307d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57c0527f7914120af7d94e1be9592c8" is_pre_election: true
I20260809 20:17:19.989214 8031 raft_consensus.cc:482] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:17:19.989328 8031 raft_consensus.cc:3037] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:19.989388 7780 raft_consensus.cc:2454] T c65b548fff22408aaa1b7147f6a1d07f P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b02be2e464f84631af2c4dc241d307d5 in term 0.
I20260809 20:17:19.991737 8031 raft_consensus.cc:504] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:19.992260 8031 leader_election.cc:288] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5 [CANDIDATE]: Term 1 election: Requested vote from peers cf9e170e36ea473ab8dedfbf3e237ca5 (127.5.61.1:43907), a57c0527f7914120af7d94e1be9592c8 (127.5.61.2:36835)
I20260809 20:17:19.992784 7614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c65b548fff22408aaa1b7147f6a1d07f" candidate_uuid: "b02be2e464f84631af2c4dc241d307d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5"
I20260809 20:17:19.992837 7780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c65b548fff22408aaa1b7147f6a1d07f" candidate_uuid: "b02be2e464f84631af2c4dc241d307d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57c0527f7914120af7d94e1be9592c8"
I20260809 20:17:19.993010 7780 raft_consensus.cc:3037] T c65b548fff22408aaa1b7147f6a1d07f P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:19.993021 7614 raft_consensus.cc:3037] T c65b548fff22408aaa1b7147f6a1d07f P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:19.996119 7780 raft_consensus.cc:2454] T c65b548fff22408aaa1b7147f6a1d07f P a57c0527f7914120af7d94e1be9592c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b02be2e464f84631af2c4dc241d307d5 in term 1.
I20260809 20:17:19.996120 7614 raft_consensus.cc:2454] T c65b548fff22408aaa1b7147f6a1d07f P cf9e170e36ea473ab8dedfbf3e237ca5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b02be2e464f84631af2c4dc241d307d5 in term 1.
I20260809 20:17:19.996526 7866 leader_election.cc:302] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a57c0527f7914120af7d94e1be9592c8, b02be2e464f84631af2c4dc241d307d5; no voters:
I20260809 20:17:19.996840 8031 raft_consensus.cc:2781] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:19.997313 8031 raft_consensus.cc:686] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5 [term 1 LEADER]: Becoming Leader. State: Replica: b02be2e464f84631af2c4dc241d307d5, State: Running, Role: LEADER
I20260809 20:17:19.997846 8031 consensus_queue.cc:237] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:20.001281 8055 tablet_replica.cc:439] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5: TxnStatusTablet state changed. Reason: New leader b02be2e464f84631af2c4dc241d307d5. Latest consensus state: current_term: 1 leader_uuid: "b02be2e464f84631af2c4dc241d307d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } } }
I20260809 20:17:20.001703 8055 tablet_replica.cc:442] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5: This TxnStatusTablet replica's current role is: LEADER
I20260809 20:17:20.002451 8057 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 20:17:20.002722 8057 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 20:17:20.002645 7429 catalog_manager.cc:5495] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b02be2e464f84631af2c4dc241d307d5 (127.5.61.3). New cstate: current_term: 1 leader_uuid: "b02be2e464f84631af2c4dc241d307d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260809 20:17:20.022372 8048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:20.016554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:38807 (local address 127.0.0.1:53934)
0809 20:17:20.016808 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:20.016830 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:20.016998 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:20.017467 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:20.017471 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:20.017483 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 20:17:20.017675 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:20.017683 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:20.018862 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:20.018867 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:20.020061 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:20.020073 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:20.020821 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:20.020827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:20.021013 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:20.021067 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 20:17:20.021833 (+ 766us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 20:17:20.021855 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 20:17:20.021942 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 20:17:20.022088 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 20:17:20.022410 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:20.016632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53934 (local address 127.5.61.3:38807)
0809 20:17:20.016804 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:20.016810 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:20.016911 (+ 101us) server_negotiation.cc:407] Connection header received
0809 20:17:20.017198 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:20.017202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:20.017304 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:20.017447 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:17:20.017823 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:20.018709 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:20.020217 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:20.020651 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:20.020795 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:20.021261 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:17:20.021721 (+ 460us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:17:20.021877 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:20.021977 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 20:17:20.022064 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 20:17:20.029901 7614 raft_consensus.cc:1261] T c65b548fff22408aaa1b7147f6a1d07f P cf9e170e36ea473ab8dedfbf3e237ca5 [term 1 FOLLOWER]: Refusing update from remote peer b02be2e464f84631af2c4dc241d307d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:17:20.029878 7780 raft_consensus.cc:1261] T c65b548fff22408aaa1b7147f6a1d07f P a57c0527f7914120af7d94e1be9592c8 [term 1 FOLLOWER]: Refusing update from remote peer b02be2e464f84631af2c4dc241d307d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:17:20.030819 8055 consensus_queue.cc:1035] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:17:20.031205 8031 consensus_queue.cc:1035] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:17:20.038823 8034 tablet_replica.cc:439] T c65b548fff22408aaa1b7147f6a1d07f P a57c0527f7914120af7d94e1be9592c8: TxnStatusTablet state changed. Reason: New leader b02be2e464f84631af2c4dc241d307d5. Latest consensus state: current_term: 1 leader_uuid: "b02be2e464f84631af2c4dc241d307d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } } }
I20260809 20:17:20.039179 8034 tablet_replica.cc:442] T c65b548fff22408aaa1b7147f6a1d07f P a57c0527f7914120af7d94e1be9592c8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 20:17:20.044881 8063 mvcc.cc:205] Tried to move back new op lower bound from 7316711997552275456 to 7316711997434949632. Current Snapshot: MvccSnapshot[applied={T|T < 7316711997552275456}]
I20260809 20:17:20.044903 8031 tablet_replica.cc:439] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b02be2e464f84631af2c4dc241d307d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } } }
I20260809 20:17:20.045325 8031 tablet_replica.cc:442] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5: This TxnStatusTablet replica's current role is: LEADER
I20260809 20:17:20.047436 8033 tablet_replica.cc:439] T c65b548fff22408aaa1b7147f6a1d07f P cf9e170e36ea473ab8dedfbf3e237ca5: TxnStatusTablet state changed. Reason: New leader b02be2e464f84631af2c4dc241d307d5. Latest consensus state: current_term: 1 leader_uuid: "b02be2e464f84631af2c4dc241d307d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } } }
I20260809 20:17:20.047878 8033 tablet_replica.cc:442] T c65b548fff22408aaa1b7147f6a1d07f P cf9e170e36ea473ab8dedfbf3e237ca5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 20:17:20.052243 8055 tablet_replica.cc:439] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b02be2e464f84631af2c4dc241d307d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } } }
I20260809 20:17:20.052515 8055 tablet_replica.cc:442] T c65b548fff22408aaa1b7147f6a1d07f P b02be2e464f84631af2c4dc241d307d5: This TxnStatusTablet replica's current role is: LEADER
I20260809 20:17:20.051515 8034 tablet_replica.cc:439] T c65b548fff22408aaa1b7147f6a1d07f P a57c0527f7914120af7d94e1be9592c8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b02be2e464f84631af2c4dc241d307d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } } }
I20260809 20:17:20.052623 8034 tablet_replica.cc:442] T c65b548fff22408aaa1b7147f6a1d07f P a57c0527f7914120af7d94e1be9592c8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 20:17:20.057389 8033 tablet_replica.cc:439] T c65b548fff22408aaa1b7147f6a1d07f P cf9e170e36ea473ab8dedfbf3e237ca5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b02be2e464f84631af2c4dc241d307d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } } }
I20260809 20:17:20.057694 8033 tablet_replica.cc:442] T c65b548fff22408aaa1b7147f6a1d07f P cf9e170e36ea473ab8dedfbf3e237ca5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 20:17:20.066470 8024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:20.060749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57326 (local address 127.5.61.2:36835)
0809 20:17:20.060902 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:20.060906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:20.061183 (+ 277us) server_negotiation.cc:407] Connection header received
0809 20:17:20.061459 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:20.061463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:20.061538 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:20.061683 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:17:20.062062 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:20.062865 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:20.064347 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:20.064805 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:20.064949 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:20.065402 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:17:20.065876 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:17:20.066022 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:20.066113 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 20:17:20.066199 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 20:17:20.082840 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:20.077151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:33039 (local address 127.5.61.3:38807)
0809 20:17:20.077299 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:20.077304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:20.077878 (+ 574us) server_negotiation.cc:407] Connection header received
0809 20:17:20.078066 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:20.078071 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:20.078164 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:20.078285 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:17:20.078796 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:20.079548 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:20.081073 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:20.081529 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:20.081691 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:20.082103 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:17:20.082225 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:17:20.082330 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:20.082480 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 20:17:20.082555 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 20:17:20.083276 8073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:20.077150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:38807 (local address 127.5.61.2:33039)
0809 20:17:20.077777 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:20.077792 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:20.077943 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:20.078302 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:20.078306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:20.078317 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 20:17:20.078652 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:20.078660 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:20.079715 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:20.079723 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:20.080942 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:20.080953 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:20.081746 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:20.081753 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:20.081871 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:20.081917 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 20:17:20.082338 (+ 421us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 20:17:20.082350 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 20:17:20.082459 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 20:17:20.082609 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":319,"threads_started":1}
I20260809 20:17:20.093533 7614 raft_consensus.cc:1261] T 09ad6ef7fcdb414ca34369f9db54234e P cf9e170e36ea473ab8dedfbf3e237ca5 [term 1 FOLLOWER]: Refusing update from remote peer a57c0527f7914120af7d94e1be9592c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:17:20.094014 7943 raft_consensus.cc:1261] T 09ad6ef7fcdb414ca34369f9db54234e P b02be2e464f84631af2c4dc241d307d5 [term 1 FOLLOWER]: Refusing update from remote peer a57c0527f7914120af7d94e1be9592c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:17:20.094389 8034 consensus_queue.cc:1035] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:17:20.094733 8040 consensus_queue.cc:1035] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:17:20.114965 8061 mvcc.cc:205] Tried to move back new op lower bound from 7316711997817171968 to 7316711996694802432. Current Snapshot: MvccSnapshot[applied={T|T < 7316711997817171968}]
I20260809 20:17:20.141360 8024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:20.134576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:53909 (local address 127.5.61.2:36835)
0809 20:17:20.134789 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:20.134798 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:20.135245 (+ 447us) server_negotiation.cc:407] Connection header received
0809 20:17:20.135477 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:20.135481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:20.135617 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:20.135731 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:17:20.136330 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:20.137477 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:20.139155 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:20.139811 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:20.139947 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:20.140618 (+ 671us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:17:20.140769 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:17:20.140853 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:20.141022 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 20:17:20.141106 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 20:17:20.142017 8079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:20.134508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:36835 (local address 127.5.61.3:53909)
0809 20:17:20.135136 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:20.135157 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:20.135297 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:20.135768 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:20.135772 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:20.135787 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 20:17:20.136139 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:20.136154 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:20.137614 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:20.137618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:20.138995 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:20.139008 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:20.140127 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:20.140132 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:20.140355 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:20.140421 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 20:17:20.140852 (+ 431us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 20:17:20.140865 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 20:17:20.141557 (+ 692us) client_negotiation.cc:240] Negotiation successful
0809 20:17:20.141724 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":273,"threads_started":1}
I20260809 20:17:20.182509 7427 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:17:20.182802 7427 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:17:20.186977 7427 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4: Sending DeleteTablet for 3 replicas of tablet 09ad6ef7fcdb414ca34369f9db54234e
I20260809 20:17:20.188544 7591 tablet_service.cc:1514] Processing DeleteTablet for tablet 09ad6ef7fcdb414ca34369f9db54234e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:17:20 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:47300
I20260809 20:17:20.188650 7919 tablet_service.cc:1514] Processing DeleteTablet for tablet 09ad6ef7fcdb414ca34369f9db54234e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:17:20 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:53930
I20260809 20:17:20.189317 7756 tablet_service.cc:1514] Processing DeleteTablet for tablet 09ad6ef7fcdb414ca34369f9db54234e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:17:20 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:57324
I20260809 20:17:20.190358 7427 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55736:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:17:20.190972 7427 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:17:20.204267 8086 tablet_replica.cc:330] T 09ad6ef7fcdb414ca34369f9db54234e P b02be2e464f84631af2c4dc241d307d5: stopping tablet replica
I20260809 20:17:20.205544 7918 tablet_service.cc:1467] Processing CreateTablet for tablet c677ccf4e9c44e14bc55ed244ef28b1f (DEFAULT_TABLE table=test-table [id=004cc21198de4105a46743812edb0674]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:20.205765 7590 tablet_service.cc:1467] Processing CreateTablet for tablet c677ccf4e9c44e14bc55ed244ef28b1f (DEFAULT_TABLE table=test-table [id=004cc21198de4105a46743812edb0674]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:20.206223 7918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c677ccf4e9c44e14bc55ed244ef28b1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:20.206322 7590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c677ccf4e9c44e14bc55ed244ef28b1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:20.212146 7755 tablet_service.cc:1467] Processing CreateTablet for tablet c677ccf4e9c44e14bc55ed244ef28b1f (DEFAULT_TABLE table=test-table [id=004cc21198de4105a46743812edb0674]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:20.212733 7755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c677ccf4e9c44e14bc55ed244ef28b1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:20.215621 8026 tablet_bootstrap.cc:492] T c677ccf4e9c44e14bc55ed244ef28b1f P cf9e170e36ea473ab8dedfbf3e237ca5: Bootstrap starting.
I20260809 20:17:20.217324 8026 tablet_bootstrap.cc:654] T c677ccf4e9c44e14bc55ed244ef28b1f P cf9e170e36ea473ab8dedfbf3e237ca5: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:20.222344 8027 tablet_bootstrap.cc:492] T c677ccf4e9c44e14bc55ed244ef28b1f P a57c0527f7914120af7d94e1be9592c8: Bootstrap starting.
I20260809 20:17:20.222708 8088 tablet_replica.cc:330] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8: stopping tablet replica
I20260809 20:17:20.223214 8088 raft_consensus.cc:2229] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:17:20.205271 8086 raft_consensus.cc:2229] T 09ad6ef7fcdb414ca34369f9db54234e P b02be2e464f84631af2c4dc241d307d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:17:20.224009 8086 raft_consensus.cc:2258] T 09ad6ef7fcdb414ca34369f9db54234e P b02be2e464f84631af2c4dc241d307d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:20.224255 8084 tablet_replica.cc:330] T 09ad6ef7fcdb414ca34369f9db54234e P cf9e170e36ea473ab8dedfbf3e237ca5: stopping tablet replica
I20260809 20:17:20.224323 8027 tablet_bootstrap.cc:654] T c677ccf4e9c44e14bc55ed244ef28b1f P a57c0527f7914120af7d94e1be9592c8: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:20.224733 8088 raft_consensus.cc:2258] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:20.225219 8084 raft_consensus.cc:2229] T 09ad6ef7fcdb414ca34369f9db54234e P cf9e170e36ea473ab8dedfbf3e237ca5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:17:20.225765 8084 raft_consensus.cc:2258] T 09ad6ef7fcdb414ca34369f9db54234e P cf9e170e36ea473ab8dedfbf3e237ca5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:20.228322 8025 tablet_bootstrap.cc:492] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5: Bootstrap starting.
I20260809 20:17:20.229892 8084 ts_tablet_manager.cc:1905] T 09ad6ef7fcdb414ca34369f9db54234e P cf9e170e36ea473ab8dedfbf3e237ca5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:20.230114 8088 ts_tablet_manager.cc:1905] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:20.230427 8025 tablet_bootstrap.cc:654] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:20.228327 8086 ts_tablet_manager.cc:1905] T 09ad6ef7fcdb414ca34369f9db54234e P b02be2e464f84631af2c4dc241d307d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:20.238600 8025 tablet_bootstrap.cc:492] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5: No bootstrap required, opened a new log
I20260809 20:17:20.238794 8025 ts_tablet_manager.cc:1397] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.004s
I20260809 20:17:20.239704 8026 tablet_bootstrap.cc:492] T c677ccf4e9c44e14bc55ed244ef28b1f P cf9e170e36ea473ab8dedfbf3e237ca5: No bootstrap required, opened a new log
I20260809 20:17:20.239863 8026 ts_tablet_manager.cc:1397] T c677ccf4e9c44e14bc55ed244ef28b1f P cf9e170e36ea473ab8dedfbf3e237ca5: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.006s
I20260809 20:17:20.239691 8025 raft_consensus.cc:348] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:20.239993 8025 raft_consensus.cc:374] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:20.240097 8025 raft_consensus.cc:729] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b02be2e464f84631af2c4dc241d307d5, State: Initialized, Role: FOLLOWER
I20260809 20:17:20.240379 8025 consensus_queue.cc:260] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:20.240614 8026 raft_consensus.cc:348] T c677ccf4e9c44e14bc55ed244ef28b1f P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:20.240845 8026 raft_consensus.cc:374] T c677ccf4e9c44e14bc55ed244ef28b1f P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:20.241954 8026 raft_consensus.cc:729] T c677ccf4e9c44e14bc55ed244ef28b1f P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf9e170e36ea473ab8dedfbf3e237ca5, State: Initialized, Role: FOLLOWER
I20260809 20:17:20.242295 8026 consensus_queue.cc:260] T c677ccf4e9c44e14bc55ed244ef28b1f P cf9e170e36ea473ab8dedfbf3e237ca5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:20.243424 8025 ts_tablet_manager.cc:1428] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260809 20:17:20.244935 8026 ts_tablet_manager.cc:1428] T c677ccf4e9c44e14bc55ed244ef28b1f P cf9e170e36ea473ab8dedfbf3e237ca5: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260809 20:17:20.245748 8088 ts_tablet_manager.cc:1918] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 20:17:20.245972 8088 log.cc:1196] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/09ad6ef7fcdb414ca34369f9db54234e
I20260809 20:17:20.246430 8084 ts_tablet_manager.cc:1918] T 09ad6ef7fcdb414ca34369f9db54234e P cf9e170e36ea473ab8dedfbf3e237ca5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 20:17:20.247098 8088 ts_tablet_manager.cc:1939] T 09ad6ef7fcdb414ca34369f9db54234e P a57c0527f7914120af7d94e1be9592c8: Deleting consensus metadata
I20260809 20:17:20.247311 8084 log.cc:1196] T 09ad6ef7fcdb414ca34369f9db54234e P cf9e170e36ea473ab8dedfbf3e237ca5: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/09ad6ef7fcdb414ca34369f9db54234e
I20260809 20:17:20.248776 8084 ts_tablet_manager.cc:1939] T 09ad6ef7fcdb414ca34369f9db54234e P cf9e170e36ea473ab8dedfbf3e237ca5: Deleting consensus metadata
I20260809 20:17:20.250382 7415 catalog_manager.cc:4841] TS a57c0527f7914120af7d94e1be9592c8 (127.5.61.2:36835): tablet 09ad6ef7fcdb414ca34369f9db54234e (table test-table [id=37cfd41ef22442998bc22259883d76de]) successfully deleted
I20260809 20:17:20.252089 7415 catalog_manager.cc:4841] TS cf9e170e36ea473ab8dedfbf3e237ca5 (127.5.61.1:43907): tablet 09ad6ef7fcdb414ca34369f9db54234e (table test-table [id=37cfd41ef22442998bc22259883d76de]) successfully deleted
I20260809 20:17:20.253098 8027 tablet_bootstrap.cc:492] T c677ccf4e9c44e14bc55ed244ef28b1f P a57c0527f7914120af7d94e1be9592c8: No bootstrap required, opened a new log
I20260809 20:17:20.253291 8086 ts_tablet_manager.cc:1918] T 09ad6ef7fcdb414ca34369f9db54234e P b02be2e464f84631af2c4dc241d307d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 20:17:20.253328 8027 ts_tablet_manager.cc:1397] T c677ccf4e9c44e14bc55ed244ef28b1f P a57c0527f7914120af7d94e1be9592c8: Time spent bootstrapping tablet: real 0.031s user 0.000s sys 0.010s
I20260809 20:17:20.253459 8086 log.cc:1196] T 09ad6ef7fcdb414ca34369f9db54234e P b02be2e464f84631af2c4dc241d307d5: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/09ad6ef7fcdb414ca34369f9db54234e
I20260809 20:17:20.254037 8086 ts_tablet_manager.cc:1939] T 09ad6ef7fcdb414ca34369f9db54234e P b02be2e464f84631af2c4dc241d307d5: Deleting consensus metadata
I20260809 20:17:20.254416 8027 raft_consensus.cc:348] T c677ccf4e9c44e14bc55ed244ef28b1f P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:20.254737 8027 raft_consensus.cc:374] T c677ccf4e9c44e14bc55ed244ef28b1f P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:20.254864 8027 raft_consensus.cc:729] T c677ccf4e9c44e14bc55ed244ef28b1f P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a57c0527f7914120af7d94e1be9592c8, State: Initialized, Role: FOLLOWER
I20260809 20:17:20.255200 8027 consensus_queue.cc:260] T c677ccf4e9c44e14bc55ed244ef28b1f P a57c0527f7914120af7d94e1be9592c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:20.255684 7412 catalog_manager.cc:4841] TS b02be2e464f84631af2c4dc241d307d5 (127.5.61.3:38807): tablet 09ad6ef7fcdb414ca34369f9db54234e (table test-table [id=37cfd41ef22442998bc22259883d76de]) successfully deleted
I20260809 20:17:20.255982 8027 ts_tablet_manager.cc:1428] T c677ccf4e9c44e14bc55ed244ef28b1f P a57c0527f7914120af7d94e1be9592c8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 20:17:20.300912 8074 raft_consensus.cc:482] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:17:20.301120 8074 raft_consensus.cc:504] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:20.301877 8074 leader_election.cc:288] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf9e170e36ea473ab8dedfbf3e237ca5 (127.5.61.1:43907), a57c0527f7914120af7d94e1be9592c8 (127.5.61.2:36835)
I20260809 20:17:20.302523 7614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c677ccf4e9c44e14bc55ed244ef28b1f" candidate_uuid: "b02be2e464f84631af2c4dc241d307d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" is_pre_election: true
I20260809 20:17:20.302528 7780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c677ccf4e9c44e14bc55ed244ef28b1f" candidate_uuid: "b02be2e464f84631af2c4dc241d307d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57c0527f7914120af7d94e1be9592c8" is_pre_election: true
I20260809 20:17:20.302788 7614 raft_consensus.cc:2454] T c677ccf4e9c44e14bc55ed244ef28b1f P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b02be2e464f84631af2c4dc241d307d5 in term 0.
I20260809 20:17:20.302795 7780 raft_consensus.cc:2454] T c677ccf4e9c44e14bc55ed244ef28b1f P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b02be2e464f84631af2c4dc241d307d5 in term 0.
I20260809 20:17:20.303372 7866 leader_election.cc:302] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b02be2e464f84631af2c4dc241d307d5, cf9e170e36ea473ab8dedfbf3e237ca5; no voters:
I20260809 20:17:20.303649 8074 raft_consensus.cc:2781] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:17:20.303802 8074 raft_consensus.cc:482] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:17:20.303900 8074 raft_consensus.cc:3037] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:20.306457 8074 raft_consensus.cc:504] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:20.307655 7614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c677ccf4e9c44e14bc55ed244ef28b1f" candidate_uuid: "b02be2e464f84631af2c4dc241d307d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5"
I20260809 20:17:20.307871 7614 raft_consensus.cc:3037] T c677ccf4e9c44e14bc55ed244ef28b1f P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:20.309832 7614 raft_consensus.cc:2454] T c677ccf4e9c44e14bc55ed244ef28b1f P cf9e170e36ea473ab8dedfbf3e237ca5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b02be2e464f84631af2c4dc241d307d5 in term 1.
I20260809 20:17:20.310218 8074 leader_election.cc:288] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5 [CANDIDATE]: Term 1 election: Requested vote from peers cf9e170e36ea473ab8dedfbf3e237ca5 (127.5.61.1:43907), a57c0527f7914120af7d94e1be9592c8 (127.5.61.2:36835)
I20260809 20:17:20.310285 7866 leader_election.cc:302] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b02be2e464f84631af2c4dc241d307d5, cf9e170e36ea473ab8dedfbf3e237ca5; no voters:
I20260809 20:17:20.310546 8074 raft_consensus.cc:2781] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:20.310972 7780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c677ccf4e9c44e14bc55ed244ef28b1f" candidate_uuid: "b02be2e464f84631af2c4dc241d307d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a57c0527f7914120af7d94e1be9592c8"
I20260809 20:17:20.311076 8074 raft_consensus.cc:686] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5 [term 1 LEADER]: Becoming Leader. State: Replica: b02be2e464f84631af2c4dc241d307d5, State: Running, Role: LEADER
I20260809 20:17:20.311172 7780 raft_consensus.cc:3037] T c677ccf4e9c44e14bc55ed244ef28b1f P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:20.311363 8074 consensus_queue.cc:237] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:20.316823 7780 raft_consensus.cc:2454] T c677ccf4e9c44e14bc55ed244ef28b1f P a57c0527f7914120af7d94e1be9592c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b02be2e464f84631af2c4dc241d307d5 in term 1.
I20260809 20:17:20.323352 7427 catalog_manager.cc:5495] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b02be2e464f84631af2c4dc241d307d5 (127.5.61.3). New cstate: current_term: 1 leader_uuid: "b02be2e464f84631af2c4dc241d307d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260809 20:17:20.360162 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:20.354812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53936 (local address 127.5.61.3:38807)
0809 20:17:20.354983 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:20.354989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:20.355218 (+ 229us) server_negotiation.cc:407] Connection header received
0809 20:17:20.355435 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:20.355440 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:20.355513 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:20.355668 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:17:20.356041 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:20.356798 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:20.358184 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:20.358703 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:20.358853 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:20.359314 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:17:20.359439 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:17:20.359539 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:20.359724 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 20:17:20.359803 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 20:17:20.379696 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:20.374805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:43451 (local address 127.5.61.3:38807)
0809 20:17:20.374980 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:20.374986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:20.375002 (+ 16us) server_negotiation.cc:407] Connection header received
0809 20:17:20.375067 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:20.375070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:20.375124 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:20.375235 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:17:20.375655 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:20.376554 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:20.377911 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:20.378348 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:20.378491 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:20.378912 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:17:20.379005 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:17:20.379096 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:20.379221 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 20:17:20.379284 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":53}
I20260809 20:17:20.379714 8079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:20.374511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:38807 (local address 127.5.61.3:43451)
0809 20:17:20.374710 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:20.374732 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:20.374848 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:20.375245 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:20.375249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:20.375264 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 20:17:20.375494 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:20.375502 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:20.376757 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:20.376762 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:20.377776 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:20.377786 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:20.378497 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:20.378502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:20.378707 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:20.378742 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 20:17:20.379096 (+ 354us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 20:17:20.379108 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 20:17:20.379197 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 20:17:20.379320 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":2}
I20260809 20:17:20.389882 7780 raft_consensus.cc:1261] T c677ccf4e9c44e14bc55ed244ef28b1f P a57c0527f7914120af7d94e1be9592c8 [term 1 FOLLOWER]: Refusing update from remote peer b02be2e464f84631af2c4dc241d307d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:17:20.390257 7614 raft_consensus.cc:1261] T c677ccf4e9c44e14bc55ed244ef28b1f P cf9e170e36ea473ab8dedfbf3e237ca5 [term 1 FOLLOWER]: Refusing update from remote peer b02be2e464f84631af2c4dc241d307d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:17:20.390547 8074 consensus_queue.cc:1035] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:17:20.392077 8074 consensus_queue.cc:1035] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:17:20.412014 8064 mvcc.cc:205] Tried to move back new op lower bound from 7316711999030992896 to 7316711998718623744. Current Snapshot: MvccSnapshot[applied={T|T < 7316711999030992896}]
W20260809 20:17:20.432716 8061 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 20:17:22.031035 7427 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:17:22.031265 7427 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:17:22.035223 7427 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4: Sending DeleteTablet for 3 replicas of tablet c677ccf4e9c44e14bc55ed244ef28b1f
I20260809 20:17:22.036136 7919 tablet_service.cc:1514] Processing DeleteTablet for tablet c677ccf4e9c44e14bc55ed244ef28b1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:17:22 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:53930
I20260809 20:17:22.036149 7591 tablet_service.cc:1514] Processing DeleteTablet for tablet c677ccf4e9c44e14bc55ed244ef28b1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:17:22 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:47300
I20260809 20:17:22.036218 7755 tablet_service.cc:1514] Processing DeleteTablet for tablet c677ccf4e9c44e14bc55ed244ef28b1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:17:22 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:57324
I20260809 20:17:22.048635 8121 tablet_replica.cc:330] T c677ccf4e9c44e14bc55ed244ef28b1f P cf9e170e36ea473ab8dedfbf3e237ca5: stopping tablet replica
I20260809 20:17:22.048910 8121 raft_consensus.cc:2229] T c677ccf4e9c44e14bc55ed244ef28b1f P cf9e170e36ea473ab8dedfbf3e237ca5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:17:22.049229 8121 raft_consensus.cc:2258] T c677ccf4e9c44e14bc55ed244ef28b1f P cf9e170e36ea473ab8dedfbf3e237ca5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:22.049969 8122 tablet_replica.cc:330] T c677ccf4e9c44e14bc55ed244ef28b1f P a57c0527f7914120af7d94e1be9592c8: stopping tablet replica
I20260809 20:17:22.050275 8122 raft_consensus.cc:2229] T c677ccf4e9c44e14bc55ed244ef28b1f P a57c0527f7914120af7d94e1be9592c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:17:22.050575 8122 raft_consensus.cc:2258] T c677ccf4e9c44e14bc55ed244ef28b1f P a57c0527f7914120af7d94e1be9592c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:22.050727 8121 ts_tablet_manager.cc:1905] T c677ccf4e9c44e14bc55ed244ef28b1f P cf9e170e36ea473ab8dedfbf3e237ca5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:22.051862 8122 ts_tablet_manager.cc:1905] T c677ccf4e9c44e14bc55ed244ef28b1f P a57c0527f7914120af7d94e1be9592c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:22.051736 8120 tablet_replica.cc:330] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5: stopping tablet replica
I20260809 20:17:22.052174 8120 raft_consensus.cc:2229] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:17:22.052772 8120 raft_consensus.cc:2258] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:22.056424 8120 ts_tablet_manager.cc:1905] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:22.059124 8122 ts_tablet_manager.cc:1918] T c677ccf4e9c44e14bc55ed244ef28b1f P a57c0527f7914120af7d94e1be9592c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 20:17:22.059284 8122 log.cc:1196] T c677ccf4e9c44e14bc55ed244ef28b1f P a57c0527f7914120af7d94e1be9592c8: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/c677ccf4e9c44e14bc55ed244ef28b1f
I20260809 20:17:22.059286 8121 ts_tablet_manager.cc:1918] T c677ccf4e9c44e14bc55ed244ef28b1f P cf9e170e36ea473ab8dedfbf3e237ca5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 20:17:22.059424 8121 log.cc:1196] T c677ccf4e9c44e14bc55ed244ef28b1f P cf9e170e36ea473ab8dedfbf3e237ca5: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/c677ccf4e9c44e14bc55ed244ef28b1f
I20260809 20:17:22.059863 8122 ts_tablet_manager.cc:1939] T c677ccf4e9c44e14bc55ed244ef28b1f P a57c0527f7914120af7d94e1be9592c8: Deleting consensus metadata
I20260809 20:17:22.059900 8121 ts_tablet_manager.cc:1939] T c677ccf4e9c44e14bc55ed244ef28b1f P cf9e170e36ea473ab8dedfbf3e237ca5: Deleting consensus metadata
I20260809 20:17:22.061072 7415 catalog_manager.cc:4841] TS cf9e170e36ea473ab8dedfbf3e237ca5 (127.5.61.1:43907): tablet c677ccf4e9c44e14bc55ed244ef28b1f (table test-table [id=004cc21198de4105a46743812edb0674]) successfully deleted
I20260809 20:17:22.061342 7415 catalog_manager.cc:4841] TS a57c0527f7914120af7d94e1be9592c8 (127.5.61.2:36835): tablet c677ccf4e9c44e14bc55ed244ef28b1f (table test-table [id=004cc21198de4105a46743812edb0674]) successfully deleted
I20260809 20:17:22.064263 8120 ts_tablet_manager.cc:1918] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 20:17:22.064424 8120 log.cc:1196] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/c677ccf4e9c44e14bc55ed244ef28b1f
I20260809 20:17:22.064905 8120 ts_tablet_manager.cc:1939] T c677ccf4e9c44e14bc55ed244ef28b1f P b02be2e464f84631af2c4dc241d307d5: Deleting consensus metadata
I20260809 20:17:22.065891 7412 catalog_manager.cc:4841] TS b02be2e464f84631af2c4dc241d307d5 (127.5.61.3:38807): tablet c677ccf4e9c44e14bc55ed244ef28b1f (table test-table [id=004cc21198de4105a46743812edb0674]) successfully deleted
Aug 09 20:17:22 dist-test-slave-6k22 krb5kdc[7376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306642, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 20:17:22 dist-test-slave-6k22 krb5kdc[7376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306642, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:22.101855 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:22.080973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55768 (local address 127.5.61.62:46769)
0809 20:17:22.081832 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:22.081838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:22.081862 (+ 24us) server_negotiation.cc:407] Connection header received
0809 20:17:22.081927 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:22.081934 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:22.082007 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:22.082114 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:22.083424 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:22.084650 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:22.085912 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:22.086552 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:22.086680 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:22.095840 (+ 9160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:22.095866 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:22.095872 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:22.095913 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:22.099373 (+ 3460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:22.100101 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:22.100106 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:22.100110 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:22.100288 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:22.100632 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:22.100639 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:22.100645 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:22.100945 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:22.101131 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:22.101446 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 20:17:22.101604 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":730,"thread_start_us":563,"threads_started":1}
I20260809 20:17:22.105707 7427 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:17:22.106379 7427 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:17:22.112640 7591 tablet_service.cc:1467] Processing CreateTablet for tablet 1761126fbdb7494cbd52d15266717a49 (DEFAULT_TABLE table=test-table [id=ddb83d4fb543484bba55b7fec3c9b343]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:22.112619 7755 tablet_service.cc:1467] Processing CreateTablet for tablet 1761126fbdb7494cbd52d15266717a49 (DEFAULT_TABLE table=test-table [id=ddb83d4fb543484bba55b7fec3c9b343]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:22.112875 7919 tablet_service.cc:1467] Processing CreateTablet for tablet 1761126fbdb7494cbd52d15266717a49 (DEFAULT_TABLE table=test-table [id=ddb83d4fb543484bba55b7fec3c9b343]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:22.113066 7591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1761126fbdb7494cbd52d15266717a49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:22.113086 7755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1761126fbdb7494cbd52d15266717a49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:22.113327 7919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1761126fbdb7494cbd52d15266717a49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:22.119292 8131 tablet_bootstrap.cc:492] T 1761126fbdb7494cbd52d15266717a49 P b02be2e464f84631af2c4dc241d307d5: Bootstrap starting.
I20260809 20:17:22.120049 8132 tablet_bootstrap.cc:492] T 1761126fbdb7494cbd52d15266717a49 P cf9e170e36ea473ab8dedfbf3e237ca5: Bootstrap starting.
I20260809 20:17:22.121356 8131 tablet_bootstrap.cc:654] T 1761126fbdb7494cbd52d15266717a49 P b02be2e464f84631af2c4dc241d307d5: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:22.122054 8132 tablet_bootstrap.cc:654] T 1761126fbdb7494cbd52d15266717a49 P cf9e170e36ea473ab8dedfbf3e237ca5: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:22.124150 8133 tablet_bootstrap.cc:492] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8: Bootstrap starting.
I20260809 20:17:22.124634 8132 tablet_bootstrap.cc:492] T 1761126fbdb7494cbd52d15266717a49 P cf9e170e36ea473ab8dedfbf3e237ca5: No bootstrap required, opened a new log
I20260809 20:17:22.124781 8132 ts_tablet_manager.cc:1397] T 1761126fbdb7494cbd52d15266717a49 P cf9e170e36ea473ab8dedfbf3e237ca5: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260809 20:17:22.125633 8132 raft_consensus.cc:348] T 1761126fbdb7494cbd52d15266717a49 P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:22.125905 8133 tablet_bootstrap.cc:654] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:22.125888 8132 raft_consensus.cc:374] T 1761126fbdb7494cbd52d15266717a49 P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:22.125983 8132 raft_consensus.cc:729] T 1761126fbdb7494cbd52d15266717a49 P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf9e170e36ea473ab8dedfbf3e237ca5, State: Initialized, Role: FOLLOWER
I20260809 20:17:22.126222 8132 consensus_queue.cc:260] T 1761126fbdb7494cbd52d15266717a49 P cf9e170e36ea473ab8dedfbf3e237ca5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:22.127158 8132 ts_tablet_manager.cc:1428] T 1761126fbdb7494cbd52d15266717a49 P cf9e170e36ea473ab8dedfbf3e237ca5: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 20:17:22.128312 8131 tablet_bootstrap.cc:492] T 1761126fbdb7494cbd52d15266717a49 P b02be2e464f84631af2c4dc241d307d5: No bootstrap required, opened a new log
I20260809 20:17:22.128459 8131 ts_tablet_manager.cc:1397] T 1761126fbdb7494cbd52d15266717a49 P b02be2e464f84631af2c4dc241d307d5: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.004s
I20260809 20:17:22.128947 8133 tablet_bootstrap.cc:492] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8: No bootstrap required, opened a new log
I20260809 20:17:22.129084 8133 ts_tablet_manager.cc:1397] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260809 20:17:22.129199 8131 raft_consensus.cc:348] T 1761126fbdb7494cbd52d15266717a49 P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:22.129416 8131 raft_consensus.cc:374] T 1761126fbdb7494cbd52d15266717a49 P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:22.129519 8131 raft_consensus.cc:729] T 1761126fbdb7494cbd52d15266717a49 P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b02be2e464f84631af2c4dc241d307d5, State: Initialized, Role: FOLLOWER
I20260809 20:17:22.129823 8131 consensus_queue.cc:260] T 1761126fbdb7494cbd52d15266717a49 P b02be2e464f84631af2c4dc241d307d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:22.129922 8133 raft_consensus.cc:348] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:22.130147 8133 raft_consensus.cc:374] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:22.130256 8133 raft_consensus.cc:729] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a57c0527f7914120af7d94e1be9592c8, State: Initialized, Role: FOLLOWER
I20260809 20:17:22.130383 8131 ts_tablet_manager.cc:1428] T 1761126fbdb7494cbd52d15266717a49 P b02be2e464f84631af2c4dc241d307d5: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 20:17:22.130548 8133 consensus_queue.cc:260] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:22.131539 8133 ts_tablet_manager.cc:1428] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 20:17:22.161366 8139 raft_consensus.cc:482] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:17:22.161628 8139 raft_consensus.cc:504] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:22.162303 8139 leader_election.cc:288] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf9e170e36ea473ab8dedfbf3e237ca5 (127.5.61.1:43907), b02be2e464f84631af2c4dc241d307d5 (127.5.61.3:38807)
I20260809 20:17:22.162889 7943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1761126fbdb7494cbd52d15266717a49" candidate_uuid: "a57c0527f7914120af7d94e1be9592c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b02be2e464f84631af2c4dc241d307d5" is_pre_election: true
I20260809 20:17:22.162902 7617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1761126fbdb7494cbd52d15266717a49" candidate_uuid: "a57c0527f7914120af7d94e1be9592c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" is_pre_election: true
I20260809 20:17:22.163125 7943 raft_consensus.cc:2454] T 1761126fbdb7494cbd52d15266717a49 P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a57c0527f7914120af7d94e1be9592c8 in term 0.
I20260809 20:17:22.163136 7617 raft_consensus.cc:2454] T 1761126fbdb7494cbd52d15266717a49 P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a57c0527f7914120af7d94e1be9592c8 in term 0.
I20260809 20:17:22.163695 7699 leader_election.cc:302] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a57c0527f7914120af7d94e1be9592c8, b02be2e464f84631af2c4dc241d307d5; no voters:
I20260809 20:17:22.164048 8139 raft_consensus.cc:2781] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:17:22.164219 8139 raft_consensus.cc:482] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:17:22.164332 8139 raft_consensus.cc:3037] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:22.166380 8139 raft_consensus.cc:504] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:22.166942 8139 leader_election.cc:288] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8 [CANDIDATE]: Term 1 election: Requested vote from peers cf9e170e36ea473ab8dedfbf3e237ca5 (127.5.61.1:43907), b02be2e464f84631af2c4dc241d307d5 (127.5.61.3:38807)
I20260809 20:17:22.167330 7617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1761126fbdb7494cbd52d15266717a49" candidate_uuid: "a57c0527f7914120af7d94e1be9592c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5"
I20260809 20:17:22.167485 7943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1761126fbdb7494cbd52d15266717a49" candidate_uuid: "a57c0527f7914120af7d94e1be9592c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b02be2e464f84631af2c4dc241d307d5"
I20260809 20:17:22.167593 7617 raft_consensus.cc:3037] T 1761126fbdb7494cbd52d15266717a49 P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:22.167694 7943 raft_consensus.cc:3037] T 1761126fbdb7494cbd52d15266717a49 P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:22.170671 7617 raft_consensus.cc:2454] T 1761126fbdb7494cbd52d15266717a49 P cf9e170e36ea473ab8dedfbf3e237ca5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a57c0527f7914120af7d94e1be9592c8 in term 1.
I20260809 20:17:22.170673 7943 raft_consensus.cc:2454] T 1761126fbdb7494cbd52d15266717a49 P b02be2e464f84631af2c4dc241d307d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a57c0527f7914120af7d94e1be9592c8 in term 1.
I20260809 20:17:22.171047 7699 leader_election.cc:302] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a57c0527f7914120af7d94e1be9592c8, b02be2e464f84631af2c4dc241d307d5, cf9e170e36ea473ab8dedfbf3e237ca5; no voters:
I20260809 20:17:22.171317 8139 raft_consensus.cc:2781] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:22.171844 8139 raft_consensus.cc:686] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8 [term 1 LEADER]: Becoming Leader. State: Replica: a57c0527f7914120af7d94e1be9592c8, State: Running, Role: LEADER
I20260809 20:17:22.172327 8139 consensus_queue.cc:237] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:22.177384 7427 catalog_manager.cc:5495] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a57c0527f7914120af7d94e1be9592c8 (127.5.61.2). New cstate: current_term: 1 leader_uuid: "a57c0527f7914120af7d94e1be9592c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260809 20:17:22.212728 8143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:22.207622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57332 (local address 127.5.61.2:36835)
0809 20:17:22.208217 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:22.208221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:22.208240 (+ 19us) server_negotiation.cc:407] Connection header received
0809 20:17:22.208288 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:22.208292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:22.208347 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:22.208455 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:17:22.208870 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:22.209592 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:22.210893 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:22.211317 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:22.211449 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:22.211961 (+ 512us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:17:22.212069 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:17:22.212168 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:22.212340 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 20:17:22.212416 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":323,"threads_started":1}
I20260809 20:17:22.221549 7617 raft_consensus.cc:1261] T 1761126fbdb7494cbd52d15266717a49 P cf9e170e36ea473ab8dedfbf3e237ca5 [term 1 FOLLOWER]: Refusing update from remote peer a57c0527f7914120af7d94e1be9592c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:17:22.221552 7943 raft_consensus.cc:1261] T 1761126fbdb7494cbd52d15266717a49 P b02be2e464f84631af2c4dc241d307d5 [term 1 FOLLOWER]: Refusing update from remote peer a57c0527f7914120af7d94e1be9592c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:17:22.222090 8140 consensus_queue.cc:1035] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:17:22.222441 8139 consensus_queue.cc:1035] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:17:22.269412 7429 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:17:22.269661 7429 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:17:22.272475 7429 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4: Sending DeleteTablet for 3 replicas of tablet 1761126fbdb7494cbd52d15266717a49
I20260809 20:17:22.273221 7591 tablet_service.cc:1514] Processing DeleteTablet for tablet 1761126fbdb7494cbd52d15266717a49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:17:22 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:47300
I20260809 20:17:22.273277 7755 tablet_service.cc:1514] Processing DeleteTablet for tablet 1761126fbdb7494cbd52d15266717a49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:17:22 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:57324
I20260809 20:17:22.273581 8122 tablet_replica.cc:330] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8: stopping tablet replica
I20260809 20:17:22.273593 7919 tablet_service.cc:1514] Processing DeleteTablet for tablet 1761126fbdb7494cbd52d15266717a49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:17:22 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:53930
I20260809 20:17:22.273820 8121 tablet_replica.cc:330] T 1761126fbdb7494cbd52d15266717a49 P cf9e170e36ea473ab8dedfbf3e237ca5: stopping tablet replica
I20260809 20:17:22.273813 8122 raft_consensus.cc:2229] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:17:22.273881 8120 tablet_replica.cc:330] T 1761126fbdb7494cbd52d15266717a49 P b02be2e464f84631af2c4dc241d307d5: stopping tablet replica
I20260809 20:17:22.274070 8120 raft_consensus.cc:2229] T 1761126fbdb7494cbd52d15266717a49 P b02be2e464f84631af2c4dc241d307d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:17:22.274056 8121 raft_consensus.cc:2229] T 1761126fbdb7494cbd52d15266717a49 P cf9e170e36ea473ab8dedfbf3e237ca5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:17:22.274137 8122 raft_consensus.cc:2258] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:22.274252 8120 raft_consensus.cc:2258] T 1761126fbdb7494cbd52d15266717a49 P b02be2e464f84631af2c4dc241d307d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:22.274258 8121 raft_consensus.cc:2258] T 1761126fbdb7494cbd52d15266717a49 P cf9e170e36ea473ab8dedfbf3e237ca5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:22.275233 8122 ts_tablet_manager.cc:1905] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:22.275262 8121 ts_tablet_manager.cc:1905] T 1761126fbdb7494cbd52d15266717a49 P cf9e170e36ea473ab8dedfbf3e237ca5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:22.275290 8120 ts_tablet_manager.cc:1905] T 1761126fbdb7494cbd52d15266717a49 P b02be2e464f84631af2c4dc241d307d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:22.280345 8121 ts_tablet_manager.cc:1918] T 1761126fbdb7494cbd52d15266717a49 P cf9e170e36ea473ab8dedfbf3e237ca5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 20:17:22.280524 8121 log.cc:1196] T 1761126fbdb7494cbd52d15266717a49 P cf9e170e36ea473ab8dedfbf3e237ca5: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/1761126fbdb7494cbd52d15266717a49
I20260809 20:17:22.280907 8121 ts_tablet_manager.cc:1939] T 1761126fbdb7494cbd52d15266717a49 P cf9e170e36ea473ab8dedfbf3e237ca5: Deleting consensus metadata
I20260809 20:17:22.281798 8122 ts_tablet_manager.cc:1918] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 20:17:22.281963 8122 log.cc:1196] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/1761126fbdb7494cbd52d15266717a49
I20260809 20:17:22.281989 7415 catalog_manager.cc:4841] TS cf9e170e36ea473ab8dedfbf3e237ca5 (127.5.61.1:43907): tablet 1761126fbdb7494cbd52d15266717a49 (table test-table [id=ddb83d4fb543484bba55b7fec3c9b343]) successfully deleted
I20260809 20:17:22.282114 7429 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 20:17:22.282359 8122 ts_tablet_manager.cc:1939] T 1761126fbdb7494cbd52d15266717a49 P a57c0527f7914120af7d94e1be9592c8: Deleting consensus metadata
W20260809 20:17:22.282714 7429 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:17:22.283344 7415 catalog_manager.cc:4841] TS a57c0527f7914120af7d94e1be9592c8 (127.5.61.2:36835): tablet 1761126fbdb7494cbd52d15266717a49 (table test-table [id=ddb83d4fb543484bba55b7fec3c9b343]) successfully deleted
I20260809 20:17:22.283399 8120 ts_tablet_manager.cc:1918] T 1761126fbdb7494cbd52d15266717a49 P b02be2e464f84631af2c4dc241d307d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 20:17:22.283535 8120 log.cc:1196] T 1761126fbdb7494cbd52d15266717a49 P b02be2e464f84631af2c4dc241d307d5: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/1761126fbdb7494cbd52d15266717a49
I20260809 20:17:22.283987 8120 ts_tablet_manager.cc:1939] T 1761126fbdb7494cbd52d15266717a49 P b02be2e464f84631af2c4dc241d307d5: Deleting consensus metadata
I20260809 20:17:22.285012 7412 catalog_manager.cc:4841] TS b02be2e464f84631af2c4dc241d307d5 (127.5.61.3:38807): tablet 1761126fbdb7494cbd52d15266717a49 (table test-table [id=ddb83d4fb543484bba55b7fec3c9b343]) successfully deleted
I20260809 20:17:22.288978 7591 tablet_service.cc:1467] Processing CreateTablet for tablet eb9f72779c51429a91d70474b2a68eb3 (DEFAULT_TABLE table=test-table [id=b1b9a4d67ef846b4b737998971784400]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:22.289040 7755 tablet_service.cc:1467] Processing CreateTablet for tablet eb9f72779c51429a91d70474b2a68eb3 (DEFAULT_TABLE table=test-table [id=b1b9a4d67ef846b4b737998971784400]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:22.289151 7919 tablet_service.cc:1467] Processing CreateTablet for tablet eb9f72779c51429a91d70474b2a68eb3 (DEFAULT_TABLE table=test-table [id=b1b9a4d67ef846b4b737998971784400]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:22.289419 7755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb9f72779c51429a91d70474b2a68eb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:22.289428 7591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb9f72779c51429a91d70474b2a68eb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:22.289619 7919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb9f72779c51429a91d70474b2a68eb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:22.294063 8131 tablet_bootstrap.cc:492] T eb9f72779c51429a91d70474b2a68eb3 P b02be2e464f84631af2c4dc241d307d5: Bootstrap starting.
I20260809 20:17:22.295353 8131 tablet_bootstrap.cc:654] T eb9f72779c51429a91d70474b2a68eb3 P b02be2e464f84631af2c4dc241d307d5: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:22.295832 8132 tablet_bootstrap.cc:492] T eb9f72779c51429a91d70474b2a68eb3 P cf9e170e36ea473ab8dedfbf3e237ca5: Bootstrap starting.
I20260809 20:17:22.296640 8133 tablet_bootstrap.cc:492] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8: Bootstrap starting.
I20260809 20:17:22.297289 8131 tablet_bootstrap.cc:492] T eb9f72779c51429a91d70474b2a68eb3 P b02be2e464f84631af2c4dc241d307d5: No bootstrap required, opened a new log
I20260809 20:17:22.297468 8131 ts_tablet_manager.cc:1397] T eb9f72779c51429a91d70474b2a68eb3 P b02be2e464f84631af2c4dc241d307d5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 20:17:22.297497 8132 tablet_bootstrap.cc:654] T eb9f72779c51429a91d70474b2a68eb3 P cf9e170e36ea473ab8dedfbf3e237ca5: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:22.298313 8133 tablet_bootstrap.cc:654] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:22.298437 8131 raft_consensus.cc:348] T eb9f72779c51429a91d70474b2a68eb3 P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:22.298761 8131 raft_consensus.cc:374] T eb9f72779c51429a91d70474b2a68eb3 P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:22.298864 8131 raft_consensus.cc:729] T eb9f72779c51429a91d70474b2a68eb3 P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b02be2e464f84631af2c4dc241d307d5, State: Initialized, Role: FOLLOWER
I20260809 20:17:22.299116 8131 consensus_queue.cc:260] T eb9f72779c51429a91d70474b2a68eb3 P b02be2e464f84631af2c4dc241d307d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:22.299701 8131 ts_tablet_manager.cc:1428] T eb9f72779c51429a91d70474b2a68eb3 P b02be2e464f84631af2c4dc241d307d5: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260809 20:17:22.299880 8132 tablet_bootstrap.cc:492] T eb9f72779c51429a91d70474b2a68eb3 P cf9e170e36ea473ab8dedfbf3e237ca5: No bootstrap required, opened a new log
I20260809 20:17:22.300129 8132 ts_tablet_manager.cc:1397] T eb9f72779c51429a91d70474b2a68eb3 P cf9e170e36ea473ab8dedfbf3e237ca5: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.003s
I20260809 20:17:22.300925 8132 raft_consensus.cc:348] T eb9f72779c51429a91d70474b2a68eb3 P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:22.301146 8132 raft_consensus.cc:374] T eb9f72779c51429a91d70474b2a68eb3 P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:22.301232 8132 raft_consensus.cc:729] T eb9f72779c51429a91d70474b2a68eb3 P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf9e170e36ea473ab8dedfbf3e237ca5, State: Initialized, Role: FOLLOWER
I20260809 20:17:22.301481 8132 consensus_queue.cc:260] T eb9f72779c51429a91d70474b2a68eb3 P cf9e170e36ea473ab8dedfbf3e237ca5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:22.301832 8133 tablet_bootstrap.cc:492] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8: No bootstrap required, opened a new log
I20260809 20:17:22.301992 8133 ts_tablet_manager.cc:1397] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260809 20:17:22.302201 8132 ts_tablet_manager.cc:1428] T eb9f72779c51429a91d70474b2a68eb3 P cf9e170e36ea473ab8dedfbf3e237ca5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 20:17:22.302749 8133 raft_consensus.cc:348] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:22.302935 8133 raft_consensus.cc:374] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:22.303030 8133 raft_consensus.cc:729] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a57c0527f7914120af7d94e1be9592c8, State: Initialized, Role: FOLLOWER
I20260809 20:17:22.303238 8133 consensus_queue.cc:260] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:22.303759 8133 ts_tablet_manager.cc:1428] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 20:17:22.524416 8140 raft_consensus.cc:482] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:17:22.524734 8140 raft_consensus.cc:504] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:22.525533 8140 leader_election.cc:288] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf9e170e36ea473ab8dedfbf3e237ca5 (127.5.61.1:43907), b02be2e464f84631af2c4dc241d307d5 (127.5.61.3:38807)
I20260809 20:17:22.526198 7943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb9f72779c51429a91d70474b2a68eb3" candidate_uuid: "a57c0527f7914120af7d94e1be9592c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b02be2e464f84631af2c4dc241d307d5" is_pre_election: true
I20260809 20:17:22.526479 7943 raft_consensus.cc:2454] T eb9f72779c51429a91d70474b2a68eb3 P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a57c0527f7914120af7d94e1be9592c8 in term 0.
I20260809 20:17:22.526458 7617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb9f72779c51429a91d70474b2a68eb3" candidate_uuid: "a57c0527f7914120af7d94e1be9592c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" is_pre_election: true
I20260809 20:17:22.526707 7617 raft_consensus.cc:2454] T eb9f72779c51429a91d70474b2a68eb3 P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a57c0527f7914120af7d94e1be9592c8 in term 0.
I20260809 20:17:22.526904 7699 leader_election.cc:302] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a57c0527f7914120af7d94e1be9592c8, b02be2e464f84631af2c4dc241d307d5; no voters:
I20260809 20:17:22.527268 8140 raft_consensus.cc:2781] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:17:22.527374 8140 raft_consensus.cc:482] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:17:22.527487 8140 raft_consensus.cc:3037] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:22.529556 8140 raft_consensus.cc:504] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:22.530089 8140 leader_election.cc:288] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8 [CANDIDATE]: Term 1 election: Requested vote from peers cf9e170e36ea473ab8dedfbf3e237ca5 (127.5.61.1:43907), b02be2e464f84631af2c4dc241d307d5 (127.5.61.3:38807)
I20260809 20:17:22.530586 7943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb9f72779c51429a91d70474b2a68eb3" candidate_uuid: "a57c0527f7914120af7d94e1be9592c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b02be2e464f84631af2c4dc241d307d5"
I20260809 20:17:22.530649 7617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb9f72779c51429a91d70474b2a68eb3" candidate_uuid: "a57c0527f7914120af7d94e1be9592c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5"
I20260809 20:17:22.530779 7943 raft_consensus.cc:3037] T eb9f72779c51429a91d70474b2a68eb3 P b02be2e464f84631af2c4dc241d307d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:22.530870 7617 raft_consensus.cc:3037] T eb9f72779c51429a91d70474b2a68eb3 P cf9e170e36ea473ab8dedfbf3e237ca5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:22.532845 7943 raft_consensus.cc:2454] T eb9f72779c51429a91d70474b2a68eb3 P b02be2e464f84631af2c4dc241d307d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a57c0527f7914120af7d94e1be9592c8 in term 1.
I20260809 20:17:22.532856 7617 raft_consensus.cc:2454] T eb9f72779c51429a91d70474b2a68eb3 P cf9e170e36ea473ab8dedfbf3e237ca5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a57c0527f7914120af7d94e1be9592c8 in term 1.
I20260809 20:17:22.533244 7701 leader_election.cc:302] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a57c0527f7914120af7d94e1be9592c8, cf9e170e36ea473ab8dedfbf3e237ca5; no voters:
I20260809 20:17:22.533525 8140 raft_consensus.cc:2781] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:22.533704 8140 raft_consensus.cc:686] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8 [term 1 LEADER]: Becoming Leader. State: Replica: a57c0527f7914120af7d94e1be9592c8, State: Running, Role: LEADER
I20260809 20:17:22.533962 8140 consensus_queue.cc:237] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } }
I20260809 20:17:22.536669 7429 catalog_manager.cc:5495] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a57c0527f7914120af7d94e1be9592c8 (127.5.61.2). New cstate: current_term: 1 leader_uuid: "a57c0527f7914120af7d94e1be9592c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a57c0527f7914120af7d94e1be9592c8" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260809 20:17:22.615407 7943 raft_consensus.cc:1261] T eb9f72779c51429a91d70474b2a68eb3 P b02be2e464f84631af2c4dc241d307d5 [term 1 FOLLOWER]: Refusing update from remote peer a57c0527f7914120af7d94e1be9592c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:17:22.615963 8140 consensus_queue.cc:1035] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b02be2e464f84631af2c4dc241d307d5" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:17:22.616242 7617 raft_consensus.cc:1261] T eb9f72779c51429a91d70474b2a68eb3 P cf9e170e36ea473ab8dedfbf3e237ca5 [term 1 FOLLOWER]: Refusing update from remote peer a57c0527f7914120af7d94e1be9592c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:17:22.616964 8140 consensus_queue.cc:1035] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf9e170e36ea473ab8dedfbf3e237ca5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 20:17:22.633867 8099 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 20:17:24.217082 7428 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:17:24.217285 7428 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:17:24.221144 7428 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 133721837e8b4b5ead5a8e556fa6e8f4: Sending DeleteTablet for 3 replicas of tablet eb9f72779c51429a91d70474b2a68eb3
I20260809 20:17:24.221901 7591 tablet_service.cc:1514] Processing DeleteTablet for tablet eb9f72779c51429a91d70474b2a68eb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:17:24 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:47300
I20260809 20:17:24.221949 7755 tablet_service.cc:1514] Processing DeleteTablet for tablet eb9f72779c51429a91d70474b2a68eb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:17:24 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:57324
I20260809 20:17:24.221936 7919 tablet_service.cc:1514] Processing DeleteTablet for tablet eb9f72779c51429a91d70474b2a68eb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:17:24 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:53930
I20260809 20:17:24.222703 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 7509
I20260809 20:17:24.237390 8181 tablet_replica.cc:330] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8: stopping tablet replica
I20260809 20:17:24.237473 8182 tablet_replica.cc:330] T eb9f72779c51429a91d70474b2a68eb3 P b02be2e464f84631af2c4dc241d307d5: stopping tablet replica
I20260809 20:17:24.237633 8181 raft_consensus.cc:2229] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:17:24.237684 8182 raft_consensus.cc:2229] T eb9f72779c51429a91d70474b2a68eb3 P b02be2e464f84631af2c4dc241d307d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:17:24.237983 8182 raft_consensus.cc:2258] T eb9f72779c51429a91d70474b2a68eb3 P b02be2e464f84631af2c4dc241d307d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:24.238075 8181 raft_consensus.cc:2258] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:24.239069 8182 ts_tablet_manager.cc:1905] T eb9f72779c51429a91d70474b2a68eb3 P b02be2e464f84631af2c4dc241d307d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:24.239835 8181 ts_tablet_manager.cc:1905] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:24.245685 8182 ts_tablet_manager.cc:1918] T eb9f72779c51429a91d70474b2a68eb3 P b02be2e464f84631af2c4dc241d307d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 20:17:24.245813 8182 log.cc:1196] T eb9f72779c51429a91d70474b2a68eb3 P b02be2e464f84631af2c4dc241d307d5: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/eb9f72779c51429a91d70474b2a68eb3
I20260809 20:17:24.246282 8182 ts_tablet_manager.cc:1939] T eb9f72779c51429a91d70474b2a68eb3 P b02be2e464f84631af2c4dc241d307d5: Deleting consensus metadata
I20260809 20:17:24.246268 8181 ts_tablet_manager.cc:1918] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 20:17:24.246410 8181 log.cc:1196] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/eb9f72779c51429a91d70474b2a68eb3
I20260809 20:17:24.246852 8181 ts_tablet_manager.cc:1939] T eb9f72779c51429a91d70474b2a68eb3 P a57c0527f7914120af7d94e1be9592c8: Deleting consensus metadata
I20260809 20:17:24.247527 7412 catalog_manager.cc:4841] TS b02be2e464f84631af2c4dc241d307d5 (127.5.61.3:38807): tablet eb9f72779c51429a91d70474b2a68eb3 (table test-table [id=b1b9a4d67ef846b4b737998971784400]) successfully deleted
I20260809 20:17:24.248029 7415 catalog_manager.cc:4841] TS a57c0527f7914120af7d94e1be9592c8 (127.5.61.2:36835): tablet eb9f72779c51429a91d70474b2a68eb3 (table test-table [id=b1b9a4d67ef846b4b737998971784400]) successfully deleted
I20260809 20:17:24.249117 8180 tablet_replica.cc:330] T eb9f72779c51429a91d70474b2a68eb3 P cf9e170e36ea473ab8dedfbf3e237ca5: stopping tablet replica
I20260809 20:17:24.249346 8180 raft_consensus.cc:2229] T eb9f72779c51429a91d70474b2a68eb3 P cf9e170e36ea473ab8dedfbf3e237ca5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:17:24.249621 8180 raft_consensus.cc:2258] T eb9f72779c51429a91d70474b2a68eb3 P cf9e170e36ea473ab8dedfbf3e237ca5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:24.250411 8180 ts_tablet_manager.cc:1905] T eb9f72779c51429a91d70474b2a68eb3 P cf9e170e36ea473ab8dedfbf3e237ca5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:24.255940 8180 ts_tablet_manager.cc:1918] T eb9f72779c51429a91d70474b2a68eb3 P cf9e170e36ea473ab8dedfbf3e237ca5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 20:17:24.256043 8180 log.cc:1196] T eb9f72779c51429a91d70474b2a68eb3 P cf9e170e36ea473ab8dedfbf3e237ca5: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/eb9f72779c51429a91d70474b2a68eb3
I20260809 20:17:24.256462 8180 ts_tablet_manager.cc:1939] T eb9f72779c51429a91d70474b2a68eb3 P cf9e170e36ea473ab8dedfbf3e237ca5: Deleting consensus metadata
Aug 09 20:17:24 dist-test-slave-6k22 krb5kdc[7376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306642, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.61.1@KRBTEST.COM
I20260809 20:17:24.257385 7415 catalog_manager.cc:4841] TS cf9e170e36ea473ab8dedfbf3e237ca5 (127.5.61.1:43907): tablet eb9f72779c51429a91d70474b2a68eb3 (table test-table [id=b1b9a4d67ef846b4b737998971784400]) successfully deleted
I20260809 20:17:24.263541 8186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:24.224527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47310 (local address 127.5.61.1:43907)
0809 20:17:24.248229 (+ 23702us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:24.248235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:24.248423 (+ 188us) server_negotiation.cc:407] Connection header received
0809 20:17:24.248698 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:24.248706 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:24.248849 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:24.248984 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:24.250595 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:24.251843 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:24.253112 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:24.253865 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:24.254015 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:24.258275 (+ 4260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:24.258289 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:24.258304 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:24.258333 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:24.261227 (+ 2894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:24.261776 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:24.261782 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:24.261784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:24.261912 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:24.262227 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:24.262230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:24.262232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:24.262567 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:24.262838 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:24.263115 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 20:17:24.263285 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23515,"thread_start_us":24014,"threads_started":1}
W20260809 20:17:24.264295 7666 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47310
W20260809 20:17:24.264947 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 7509: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 20:17:24.265174 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 7509
I20260809 20:17:24.287292 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 7676
Aug 09 20:17:24 dist-test-slave-6k22 krb5kdc[7376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306642, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.61.2@KRBTEST.COM
I20260809 20:17:24.301753 8189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:24.288103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57338 (local address 127.5.61.2:36835)
0809 20:17:24.288595 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:24.288599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:24.288615 (+ 16us) server_negotiation.cc:407] Connection header received
0809 20:17:24.288670 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:24.288674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:24.288732 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:24.288844 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:24.290052 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:24.290784 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:24.292085 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:24.292480 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:24.292569 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:24.296625 (+ 4056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:24.296637 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:24.296652 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:24.296686 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:24.299318 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:24.299983 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:24.299990 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:24.299992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:24.300120 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:24.300444 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:24.300447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:24.300449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:24.300786 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:24.301011 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:24.301197 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 20:17:24.301380 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":220,"threads_started":1}
W20260809 20:17:24.302461 7830 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57338
W20260809 20:17:24.303120 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 7676: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 20:17:24.303273 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 7676
I20260809 20:17:24.332136 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 7842
Aug 09 20:17:24 dist-test-slave-6k22 krb5kdc[7376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306642, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.61.3@KRBTEST.COM
I20260809 20:17:24.346982 8191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:24.332883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53952 (local address 127.5.61.3:38807)
0809 20:17:24.333503 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:24.333508 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:24.333524 (+ 16us) server_negotiation.cc:407] Connection header received
0809 20:17:24.333579 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:24.333583 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:24.333641 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:24.333731 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:24.335020 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:24.335852 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:24.337006 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:24.337420 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:24.337509 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:24.341666 (+ 4157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:24.341680 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:24.341720 (+ 40us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:24.341757 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:24.344724 (+ 2967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:24.345290 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:24.345297 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:24.345299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:24.345423 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:24.345739 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:24.345742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:24.345744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:24.346098 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:24.346312 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:24.346534 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 20:17:24.346724 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":342,"threads_started":1}
W20260809 20:17:24.347729 7997 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53952
W20260809 20:17:24.348395 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 7842: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 20:17:24.348585 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 7842
I20260809 20:17:24.381358 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 7391
I20260809 20:17:24.382583 7475 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:24.498996 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 7391
2026-08-09T20:17:24Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8379 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:17:24 dist-test-slave-6k22 krb5kdc[8196](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:17:24 dist-test-slave-6k22 krb5kdc[8196](info): set up 2 sockets
Aug 09 20:17:24 dist-test-slave-6k22 krb5kdc[8196](info): commencing operation
krb5kdc: starting...
W20260809 20:17:25.695302 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.000s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:17:25 dist-test-slave-6k22 krb5kdc[8196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:17:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:17:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:17:25.888633 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:38779
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:36557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:38779
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:26.219846 8210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:26.220161 8210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:26.220317 8210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:26.227186 8210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:17:26.227319 8210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:26.227461 8210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:26.227558 8210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:17:26.227669 8210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:17:26.236977 8210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:36557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:38779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:38779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:26.238212 8210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:26.239483 8210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:26.251447 8216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:26.251487 8215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:26.251519 8218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:26.252266 8210 server_base.cc:1034] running on GCE node
I20260809 20:17:26.253059 8210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:26.254428 8210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:26.255692 8210 hybrid_clock.cc:648] HybridClock initialized: now 1786306646255606 us; error 57 us; skew 500 ppm
I20260809 20:17:26.256031 8210 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:26 dist-test-slave-6k22 krb5kdc[8196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306646, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:26.259145 8210 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:17:26.260524 8210 webserver.cc:460] Webserver started at http://127.5.61.62:38295/ using document root <none> and password file <none>
I20260809 20:17:26.261209 8210 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:26.261356 8210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:26.261673 8210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:26.265141 8210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "f3044b66d5e5482f8f44bbc55bd20cbc"
format_stamp: "Formatted at 2026-08-09 20:17:26 on dist-test-slave-6k22"
server_key: "755106db57e156e691deff74b675b0ea"
server_key_iv: "c984f31ce71b5cf9a81d6207def92c60"
server_key_version: "encryptionkey@0"
I20260809 20:17:26.265956 8210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "f3044b66d5e5482f8f44bbc55bd20cbc"
format_stamp: "Formatted at 2026-08-09 20:17:26 on dist-test-slave-6k22"
server_key: "755106db57e156e691deff74b675b0ea"
server_key_iv: "c984f31ce71b5cf9a81d6207def92c60"
server_key_version: "encryptionkey@0"
I20260809 20:17:26.270915 8210 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 20:17:26.274308 8225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:26.275624 8210 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 20:17:26.275804 8210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "f3044b66d5e5482f8f44bbc55bd20cbc"
format_stamp: "Formatted at 2026-08-09 20:17:26 on dist-test-slave-6k22"
server_key: "755106db57e156e691deff74b675b0ea"
server_key_iv: "c984f31ce71b5cf9a81d6207def92c60"
server_key_version: "encryptionkey@0"
I20260809 20:17:26.276021 8210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:26.287711 8210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:26.291121 8210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:26.291381 8210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:26.309269 8210 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:38779
I20260809 20:17:26.309330 8285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:38779 every 8 connection(s)
I20260809 20:17:26.310619 8210 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:17:26.314109 8287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:26.316399 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 8210
I20260809 20:17:26.316632 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:17:26.317466 5364 external_mini_cluster.cc:1428] Setting key 5f7b2cf17dcb7cccbbf4d55e9c5f9ac0
I20260809 20:17:26.325636 8287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3044b66d5e5482f8f44bbc55bd20cbc: Bootstrap starting.
I20260809 20:17:26.328701 8287 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3044b66d5e5482f8f44bbc55bd20cbc: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:26.329667 8287 log.cc:824] T 00000000000000000000000000000000 P f3044b66d5e5482f8f44bbc55bd20cbc: Log is configured to *not* fsync() on all Append() calls
Aug 09 20:17:26 dist-test-slave-6k22 krb5kdc[8196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:26.332479 8287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3044b66d5e5482f8f44bbc55bd20cbc: No bootstrap required, opened a new log
I20260809 20:17:26.338958 8290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:26.319596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51904 (local address 127.5.61.62:38779)
0809 20:17:26.320338 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:26.320352 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:26.320404 (+ 52us) server_negotiation.cc:407] Connection header received
0809 20:17:26.321991 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:26.322011 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:26.322371 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:26.322789 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:26.324270 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:26.325572 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:26.326848 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:26.327523 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:26.327726 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:26.331809 (+ 4083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:26.331817 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:26.331830 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:26.331863 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:26.334948 (+ 3085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:26.335698 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:26.335706 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:26.335721 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:26.335891 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:26.336279 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:26.336282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:26.336284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:26.336805 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:26.337122 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:26.337596 (+ 474us) server_negotiation.cc:299] Negotiation successful
0809 20:17:26.337995 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":331,"threads_started":1}
I20260809 20:17:26.340474 8287 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3044b66d5e5482f8f44bbc55bd20cbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3044b66d5e5482f8f44bbc55bd20cbc" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38779 } }
I20260809 20:17:26.340921 8287 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3044b66d5e5482f8f44bbc55bd20cbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:26.341030 8287 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3044b66d5e5482f8f44bbc55bd20cbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3044b66d5e5482f8f44bbc55bd20cbc, State: Initialized, Role: FOLLOWER
I20260809 20:17:26.341655 8287 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3044b66d5e5482f8f44bbc55bd20cbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3044b66d5e5482f8f44bbc55bd20cbc" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38779 } }
I20260809 20:17:26.341966 8287 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3044b66d5e5482f8f44bbc55bd20cbc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:17:26.342113 8287 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3044b66d5e5482f8f44bbc55bd20cbc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:17:26.342254 8287 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3044b66d5e5482f8f44bbc55bd20cbc [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:26.344358 8287 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3044b66d5e5482f8f44bbc55bd20cbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3044b66d5e5482f8f44bbc55bd20cbc" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38779 } }
I20260809 20:17:26.344774 8287 leader_election.cc:302] T 00000000000000000000000000000000 P f3044b66d5e5482f8f44bbc55bd20cbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3044b66d5e5482f8f44bbc55bd20cbc; no voters:
I20260809 20:17:26.345309 8287 leader_election.cc:288] T 00000000000000000000000000000000 P f3044b66d5e5482f8f44bbc55bd20cbc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:17:26.345422 8292 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3044b66d5e5482f8f44bbc55bd20cbc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:26.346131 8292 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3044b66d5e5482f8f44bbc55bd20cbc [term 1 LEADER]: Becoming Leader. State: Replica: f3044b66d5e5482f8f44bbc55bd20cbc, State: Running, Role: LEADER
I20260809 20:17:26.346658 8292 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3044b66d5e5482f8f44bbc55bd20cbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3044b66d5e5482f8f44bbc55bd20cbc" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38779 } }
I20260809 20:17:26.347339 8287 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3044b66d5e5482f8f44bbc55bd20cbc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:17:26.349548 8293 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3044b66d5e5482f8f44bbc55bd20cbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3044b66d5e5482f8f44bbc55bd20cbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3044b66d5e5482f8f44bbc55bd20cbc" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38779 } } }
I20260809 20:17:26.350031 8293 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3044b66d5e5482f8f44bbc55bd20cbc [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:26.349941 8294 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3044b66d5e5482f8f44bbc55bd20cbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3044b66d5e5482f8f44bbc55bd20cbc. Latest consensus state: current_term: 1 leader_uuid: "f3044b66d5e5482f8f44bbc55bd20cbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3044b66d5e5482f8f44bbc55bd20cbc" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38779 } } }
I20260809 20:17:26.350224 8294 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3044b66d5e5482f8f44bbc55bd20cbc [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:26.351632 8300 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:17:26.356683 8300 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:17:26.365327 8300 catalog_manager.cc:1347] Generated new cluster ID: a07c9f395b1c4bf1acd5fc8d0dab1d54
I20260809 20:17:26.365443 8300 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:17:26.374313 8300 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:17:26.375624 8300 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:17:26.382408 8300 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3044b66d5e5482f8f44bbc55bd20cbc: Generated new TSK 0
I20260809 20:17:26.383153 8300 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.61.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
I20260809 20:17:26.442474 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:38779
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:36557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:26.788192 8315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:26.788441 8315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:26.788528 8315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:26.795542 8315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:26.795665 8315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:26.795773 8315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:17:26.804989 8315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:36557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:38779
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:26.806169 8315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:26.807331 8315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:26.820111 8320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:26.820226 8324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:26.820242 8321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:26.821036 8315 server_base.cc:1034] running on GCE node
I20260809 20:17:26.821543 8315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:26.822712 8315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:26.823925 8315 hybrid_clock.cc:648] HybridClock initialized: now 1786306646823869 us; error 51 us; skew 500 ppm
I20260809 20:17:26.824239 8315 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:26 dist-test-slave-6k22 krb5kdc[8196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306646, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:26.827283 8315 init.cc:377] Logged in from keytab as kudu/127.5.61.1@KRBTEST.COM (short username kudu)
I20260809 20:17:26.828794 8315 webserver.cc:460] Webserver started at http://127.5.61.1:41785/ using document root <none> and password file <none>
I20260809 20:17:26.829449 8315 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:26.829594 8315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:26.829917 8315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:26.832916 8315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "0d05687536b5467981217203c315b92e"
format_stamp: "Formatted at 2026-08-09 20:17:26 on dist-test-slave-6k22"
server_key: "e7d8649ba583b98069ae3955508a0db1"
server_key_iv: "3fa283025fe0801156f487f562464cf5"
server_key_version: "encryptionkey@0"
I20260809 20:17:26.833978 8315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "0d05687536b5467981217203c315b92e"
format_stamp: "Formatted at 2026-08-09 20:17:26 on dist-test-slave-6k22"
server_key: "e7d8649ba583b98069ae3955508a0db1"
server_key_iv: "3fa283025fe0801156f487f562464cf5"
server_key_version: "encryptionkey@0"
I20260809 20:17:26.839291 8315 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 20:17:26.842598 8333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:26.844069 8315 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:17:26.844254 8315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "0d05687536b5467981217203c315b92e"
format_stamp: "Formatted at 2026-08-09 20:17:26 on dist-test-slave-6k22"
server_key: "e7d8649ba583b98069ae3955508a0db1"
server_key_iv: "3fa283025fe0801156f487f562464cf5"
server_key_version: "encryptionkey@0"
I20260809 20:17:26.844475 8315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:26.864969 8315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:26.868805 8315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:26.869091 8315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:26.870021 8315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:26.871299 8315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:26.871419 8315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:26.871608 8315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:26.871692 8315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260809 20:17:26.905438 8315 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:35501
I20260809 20:17:26.905479 8459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:35501 every 8 connection(s)
I20260809 20:17:26.906819 8315 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:17:26.914839 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 8315
I20260809 20:17:26.915108 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:17:26.915855 5364 external_mini_cluster.cc:1428] Setting key cdf24eb18fa993aa4384137f7aa0279b
Aug 09 20:17:26 dist-test-slave-6k22 krb5kdc[8196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306646, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:26.926622 8462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:26.909428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:44005 (local address 127.5.61.62:38779)
0809 20:17:26.909886 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:26.909891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:26.910886 (+ 995us) server_negotiation.cc:407] Connection header received
0809 20:17:26.912773 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:26.912779 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:26.912868 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:26.912983 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:26.914432 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:26.915228 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:26.917043 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:26.917484 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:26.917616 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:26.921065 (+ 3449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:26.921070 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:26.921074 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:26.921105 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:26.923669 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:26.924374 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:26.924379 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:26.924383 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:26.924505 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:26.924934 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:26.924938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:26.924942 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:26.925200 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:26.925363 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:26.926039 (+ 676us) server_negotiation.cc:299] Negotiation successful
0809 20:17:26.926263 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":250,"threads_started":1}
I20260809 20:17:26.927510 8464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:26.910008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38779 (local address 127.5.61.1:44005)
0809 20:17:26.910727 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:26.910773 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:26.912502 (+ 1729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:26.913206 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:26.913214 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:26.913687 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:26.914220 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:26.914236 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:26.915402 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:26.915413 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:26.916899 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:26.916911 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:26.917706 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:26.917714 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:26.917956 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:26.918674 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:26.918700 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:26.920849 (+ 2149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:26.923804 (+ 2955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:26.923812 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:26.923827 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:26.924243 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:26.924616 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:26.924621 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:26.924625 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:26.924812 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:26.925334 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:26.925342 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:26.925665 (+ 323us) client_negotiation.cc:769] Sending connection context
0809 20:17:26.925924 (+ 259us) client_negotiation.cc:240] Negotiation successful
0809 20:17:26.926433 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":284,"threads_started":1}
I20260809 20:17:26.929193 8460 heartbeater.cc:347] Connected to a master server at 127.5.61.62:38779
I20260809 20:17:26.929502 8460 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:26.930163 8460 heartbeater.cc:511] Master 127.5.61.62:38779 requested a full tablet report, sending...
I20260809 20:17:26.932209 8246 ts_manager.cc:194] Registered new tserver with Master: 0d05687536b5467981217203c315b92e (127.5.61.1:35501)
I20260809 20:17:26.933589 8246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.1@KRBTEST.COM'} at 127.5.61.1:44005
WARNING: no policy specified for kudu/127.5.61.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
I20260809 20:17:26.962008 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:38779
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:36557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:27.291102 8469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:27.291364 8469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:27.291484 8469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:27.298753 8469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:27.298882 8469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:27.299072 8469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:17:27.308758 8469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:36557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:38779
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:27.310014 8469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:27.311296 8469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:27.324633 8475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:27.324677 8474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:27.325105 8477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:27.325863 8469 server_base.cc:1034] running on GCE node
I20260809 20:17:27.326488 8469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:27.327410 8469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:27.328646 8469 hybrid_clock.cc:648] HybridClock initialized: now 1786306647328567 us; error 67 us; skew 500 ppm
I20260809 20:17:27.328958 8469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:27 dist-test-slave-6k22 krb5kdc[8196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306647, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:27.337047 8469 init.cc:377] Logged in from keytab as kudu/127.5.61.2@KRBTEST.COM (short username kudu)
I20260809 20:17:27.338531 8469 webserver.cc:460] Webserver started at http://127.5.61.2:45019/ using document root <none> and password file <none>
I20260809 20:17:27.339259 8469 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:27.339409 8469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:27.339776 8469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:27.342667 8469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "f9fb999c6fc34f119cda478860ca1be3"
format_stamp: "Formatted at 2026-08-09 20:17:27 on dist-test-slave-6k22"
server_key: "268f0bfdc29e742ac09c212c6994aedf"
server_key_iv: "a227dc9d5a1f47b10f561a8c54c7b6d3"
server_key_version: "encryptionkey@0"
I20260809 20:17:27.343487 8469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "f9fb999c6fc34f119cda478860ca1be3"
format_stamp: "Formatted at 2026-08-09 20:17:27 on dist-test-slave-6k22"
server_key: "268f0bfdc29e742ac09c212c6994aedf"
server_key_iv: "a227dc9d5a1f47b10f561a8c54c7b6d3"
server_key_version: "encryptionkey@0"
I20260809 20:17:27.348896 8469 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260809 20:17:27.352523 8484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:27.353778 8469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 20:17:27.353935 8469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "f9fb999c6fc34f119cda478860ca1be3"
format_stamp: "Formatted at 2026-08-09 20:17:27 on dist-test-slave-6k22"
server_key: "268f0bfdc29e742ac09c212c6994aedf"
server_key_iv: "a227dc9d5a1f47b10f561a8c54c7b6d3"
server_key_version: "encryptionkey@0"
I20260809 20:17:27.354128 8469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:27.366765 8469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:27.370734 8469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:27.370985 8469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:27.371915 8469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:27.373152 8469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:27.373226 8469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:27.373335 8469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:27.373383 8469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:27.407626 8469 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:40581
I20260809 20:17:27.407663 8613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:40581 every 8 connection(s)
I20260809 20:17:27.408914 8469 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:17:27.409849 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 8469
I20260809 20:17:27.410044 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:17:27.410593 5364 external_mini_cluster.cc:1428] Setting key 0ca521d7e8b45e00eab60b0643be84f5
Aug 09 20:17:27 dist-test-slave-6k22 krb5kdc[8196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306647, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:27.430388 8462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:27.411699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:33133 (local address 127.5.61.62:38779)
0809 20:17:27.411870 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:27.411874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:27.413312 (+ 1438us) server_negotiation.cc:407] Connection header received
0809 20:17:27.415118 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:27.415123 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:27.415218 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:27.415329 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:27.416930 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:27.417648 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:27.419351 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:27.419856 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:27.419959 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:27.424273 (+ 4314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:27.424278 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:27.424281 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:27.424312 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:27.426901 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:27.427648 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:27.427652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:27.427654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:27.427779 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:27.428375 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:27.428383 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:27.428388 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:27.428805 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:27.428992 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:27.429673 (+ 681us) server_negotiation.cc:299] Negotiation successful
0809 20:17:27.429950 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 20:17:27.431033 8616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:27.412331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38779 (local address 127.5.61.2:33133)
0809 20:17:27.413162 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:27.413206 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:27.414858 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:27.415644 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:27.415652 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:27.416178 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:27.416751 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:27.416768 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:27.417816 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:27.417824 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:27.419190 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:27.419206 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:27.420089 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:27.420098 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:27.420498 (+ 400us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:27.421240 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:27.421275 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:27.424041 (+ 2766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:27.427051 (+ 3010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:27.427060 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:27.427075 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:27.427486 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:27.427902 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:27.427907 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:27.427910 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:27.428202 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:27.428944 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:27.428949 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:27.429279 (+ 330us) client_negotiation.cc:769] Sending connection context
0809 20:17:27.429518 (+ 239us) client_negotiation.cc:240] Negotiation successful
0809 20:17:27.429903 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":362,"threads_started":1}
WARNING: no policy specified for kudu/127.5.61.3@KRBTEST.COM; defaulting to no policy
I20260809 20:17:27.432530 8614 heartbeater.cc:347] Connected to a master server at 127.5.61.62:38779
I20260809 20:17:27.432924 8614 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:27.433668 8614 heartbeater.cc:511] Master 127.5.61.62:38779 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.3@KRBTEST.COM" created.
I20260809 20:17:27.435201 8246 ts_manager.cc:194] Registered new tserver with Master: f9fb999c6fc34f119cda478860ca1be3 (127.5.61.2:40581)
I20260809 20:17:27.436038 8246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.2@KRBTEST.COM'} at 127.5.61.2:33133
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
I20260809 20:17:27.458017 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:38779
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:36557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:27.792538 8619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:27.792797 8619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:27.792888 8619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:27.800354 8619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:27.800453 8619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:27.800570 8619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:17:27.809937 8619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:36557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:38779
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:27.811064 8619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:27.812350 8619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:27.825042 8625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:27.825078 8629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:27.824988 8627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:27.826187 8619 server_base.cc:1034] running on GCE node
I20260809 20:17:27.826782 8619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:27.827701 8619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:27.828925 8619 hybrid_clock.cc:648] HybridClock initialized: now 1786306647828886 us; error 48 us; skew 500 ppm
I20260809 20:17:27.829258 8619 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:27 dist-test-slave-6k22 krb5kdc[8196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306647, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:27.832384 8619 init.cc:377] Logged in from keytab as kudu/127.5.61.3@KRBTEST.COM (short username kudu)
I20260809 20:17:27.833674 8619 webserver.cc:460] Webserver started at http://127.5.61.3:37205/ using document root <none> and password file <none>
I20260809 20:17:27.834286 8619 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:27.834389 8619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:27.834659 8619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:27.837447 8619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "c09fca1b8022424cb8955182c7064bf2"
format_stamp: "Formatted at 2026-08-09 20:17:27 on dist-test-slave-6k22"
server_key: "2b2f6c6e3883485e9acf63d81f37bc75"
server_key_iv: "53a9e98f3db227662ca11a961bfa13d1"
server_key_version: "encryptionkey@0"
I20260809 20:17:27.838181 8619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "c09fca1b8022424cb8955182c7064bf2"
format_stamp: "Formatted at 2026-08-09 20:17:27 on dist-test-slave-6k22"
server_key: "2b2f6c6e3883485e9acf63d81f37bc75"
server_key_iv: "53a9e98f3db227662ca11a961bfa13d1"
server_key_version: "encryptionkey@0"
I20260809 20:17:27.843035 8619 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 20:17:27.846477 8636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:27.847872 8619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:17:27.848014 8619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "c09fca1b8022424cb8955182c7064bf2"
format_stamp: "Formatted at 2026-08-09 20:17:27 on dist-test-slave-6k22"
server_key: "2b2f6c6e3883485e9acf63d81f37bc75"
server_key_iv: "53a9e98f3db227662ca11a961bfa13d1"
server_key_version: "encryptionkey@0"
I20260809 20:17:27.848278 8619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:27.865983 8619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:27.870004 8619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:27.870242 8619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:27.871064 8619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:27.872359 8619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:27.872426 8619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:27.872503 8619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:27.872543 8619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:27.905782 8619 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:43191
I20260809 20:17:27.905804 8765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:43191 every 8 connection(s)
I20260809 20:17:27.907341 8619 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:17:27.915930 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 8619
I20260809 20:17:27.916141 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:17:27.916563 5364 external_mini_cluster.cc:1428] Setting key 0105464412a96274b0e549f2351d965f
Aug 09 20:17:27 dist-test-slave-6k22 krb5kdc[8196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306647, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:27.926823 8462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:27.909982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:49541 (local address 127.5.61.62:38779)
0809 20:17:27.910199 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:27.910205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:27.911377 (+ 1172us) server_negotiation.cc:407] Connection header received
0809 20:17:27.913064 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:27.913069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:27.913134 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:27.913226 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:27.914617 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:27.915370 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:27.917041 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:27.917436 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:27.917593 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:27.921438 (+ 3845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:27.921443 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:27.921446 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:27.921478 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:27.923879 (+ 2401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:27.924612 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:27.924619 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:27.924627 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:27.924761 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:27.925193 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:27.925198 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:27.925200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:27.925454 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:27.925623 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:27.926264 (+ 641us) server_negotiation.cc:299] Negotiation successful
0809 20:17:27.926471 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 20:17:27.927724 8768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:27.910405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38779 (local address 127.5.61.3:49541)
0809 20:17:27.911208 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:27.911252 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:27.912816 (+ 1564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:27.913439 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:27.913447 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:27.913935 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:27.914472 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:27.914489 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:27.915503 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:27.915507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:27.916899 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:27.916912 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:27.917734 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:27.917744 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:27.917985 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:27.918757 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:27.918794 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:27.921245 (+ 2451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:27.924009 (+ 2764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:27.924017 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:27.924030 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:27.924481 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:27.924876 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:27.924881 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:27.924885 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:27.925065 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:27.925599 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:27.925606 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:27.925907 (+ 301us) client_negotiation.cc:769] Sending connection context
0809 20:17:27.926160 (+ 253us) client_negotiation.cc:240] Negotiation successful
0809 20:17:27.926619 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":352,"threads_started":1}
I20260809 20:17:27.929047 8766 heartbeater.cc:347] Connected to a master server at 127.5.61.62:38779
I20260809 20:17:27.929335 8766 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:27.930044 8766 heartbeater.cc:511] Master 127.5.61.62:38779 requested a full tablet report, sending...
I20260809 20:17:27.931468 8246 ts_manager.cc:194] Registered new tserver with Master: c09fca1b8022424cb8955182c7064bf2 (127.5.61.3:43191)
I20260809 20:17:27.932052 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 20:17:27.932442 8246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.3@KRBTEST.COM'} at 127.5.61.3:49541
I20260809 20:17:27.936404 8460 heartbeater.cc:503] Master 127.5.61.62:38779 was elected leader, sending a full tablet report...
W20260809 20:17:27.966765 8462 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:17:27.964228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51906 (local address 127.5.61.62:38779)
0809 20:17:27.964431 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:27.964454 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:27.964994 (+ 540us) server_negotiation.cc:407] Connection header received
0809 20:17:27.965152 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:27.965156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:27.965223 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:27.965317 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:27.966567 (+ 1250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51906: BlockingRecv error: recv got EOF from 127.0.0.1:51906 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 20:17:27.969931 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 8315
W20260809 20:17:27.972846 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 8315: Not authorized: Client connection negotiation failed: client connection to 127.5.61.1:35501: server requires authentication, but client does not have Kerberos credentials available
I20260809 20:17:27.973035 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 8315
I20260809 20:17:27.994855 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 8469
W20260809 20:17:27.997416 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 8469: Not authorized: Client connection negotiation failed: client connection to 127.5.61.2:40581: server requires authentication, but client does not have Kerberos credentials available
I20260809 20:17:27.997586 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 8469
I20260809 20:17:28.017941 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 8619
W20260809 20:17:28.020962 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 8619: Not authorized: Client connection negotiation failed: client connection to 127.5.61.3:43191: server requires authentication, but client does not have Kerberos credentials available
I20260809 20:17:28.021083 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 8619
I20260809 20:17:28.039376 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 8210
I20260809 20:17:28.040585 8281 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:28.142190 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 8210
2026-08-09T20:17:28Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3638 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:17:28 dist-test-slave-6k22 krb5kdc[8786](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:17:28 dist-test-slave-6k22 krb5kdc[8786](info): set up 2 sockets
Aug 09 20:17:28 dist-test-slave-6k22 krb5kdc[8786](info): commencing operation
krb5kdc: starting...
W20260809 20:17:29.310618 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:17:29 dist-test-slave-6k22 krb5kdc[8786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:17:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:17:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:17:29.512787 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:36557
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:41919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:36557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:29.830437 8801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:29.830715 8801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:29.830899 8801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:29.837924 8801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:17:29.838047 8801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:29.838150 8801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:29.838274 8801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:17:29.838388 8801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:17:29.847795 8801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:41919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:36557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:36557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:29.848985 8801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:29.850250 8801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:29.862102 8807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:29.862087 8810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:29.862119 8808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:29.862983 8801 server_base.cc:1034] running on GCE node
I20260809 20:17:29.863760 8801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:29.865031 8801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:29.866264 8801 hybrid_clock.cc:648] HybridClock initialized: now 1786306649866235 us; error 49 us; skew 500 ppm
I20260809 20:17:29.866631 8801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:29 dist-test-slave-6k22 krb5kdc[8786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306649, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:29.869704 8801 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:17:29.871088 8801 webserver.cc:460] Webserver started at http://127.5.61.62:45901/ using document root <none> and password file <none>
I20260809 20:17:29.871949 8801 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:29.872128 8801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:29.872598 8801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:29.875686 8801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "1c25ff00fbf34fea8e145b671e99f521"
format_stamp: "Formatted at 2026-08-09 20:17:29 on dist-test-slave-6k22"
server_key: "a7ebdf924c41e56491fdfc635912ddf5"
server_key_iv: "aad8c888099709d287dac7921d91876d"
server_key_version: "encryptionkey@0"
I20260809 20:17:29.876482 8801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "1c25ff00fbf34fea8e145b671e99f521"
format_stamp: "Formatted at 2026-08-09 20:17:29 on dist-test-slave-6k22"
server_key: "a7ebdf924c41e56491fdfc635912ddf5"
server_key_iv: "aad8c888099709d287dac7921d91876d"
server_key_version: "encryptionkey@0"
I20260809 20:17:29.881510 8801 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 20:17:29.884838 8819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:29.886132 8801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:17:29.886315 8801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "1c25ff00fbf34fea8e145b671e99f521"
format_stamp: "Formatted at 2026-08-09 20:17:29 on dist-test-slave-6k22"
server_key: "a7ebdf924c41e56491fdfc635912ddf5"
server_key_iv: "aad8c888099709d287dac7921d91876d"
server_key_version: "encryptionkey@0"
I20260809 20:17:29.886581 8801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:29.911248 8801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:29.914891 8801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:29.915170 8801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:29.933965 8801 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:36557
I20260809 20:17:29.933995 8879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:36557 every 8 connection(s)
I20260809 20:17:29.935379 8801 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:17:29.939431 8880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:29.941462 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 8801
I20260809 20:17:29.941684 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:17:29.942456 5364 external_mini_cluster.cc:1428] Setting key 8dc1f5b8666bcf4ebbd7d6497338f7df
I20260809 20:17:29.950948 8880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c25ff00fbf34fea8e145b671e99f521: Bootstrap starting.
I20260809 20:17:29.953935 8880 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c25ff00fbf34fea8e145b671e99f521: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:29.954886 8880 log.cc:824] T 00000000000000000000000000000000 P 1c25ff00fbf34fea8e145b671e99f521: Log is configured to *not* fsync() on all Append() calls
Aug 09 20:17:29 dist-test-slave-6k22 krb5kdc[8786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:29.957983 8880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c25ff00fbf34fea8e145b671e99f521: No bootstrap required, opened a new log
I20260809 20:17:29.963979 8883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:29.944545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45268 (local address 127.5.61.62:36557)
0809 20:17:29.945350 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:29.945361 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:29.945401 (+ 40us) server_negotiation.cc:407] Connection header received
0809 20:17:29.946991 (+ 1590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:29.947011 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:29.947385 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:29.947784 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:29.949282 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:29.950585 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:29.951932 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:29.952552 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:29.952766 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:29.956961 (+ 4195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:29.956968 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:29.956981 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:29.957018 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:29.960096 (+ 3078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:29.960878 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:29.960883 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:29.960888 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:29.961051 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:29.961424 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:29.961427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:29.961429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:29.961900 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:29.962136 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:29.962570 (+ 434us) server_negotiation.cc:299] Negotiation successful
0809 20:17:29.962941 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":362,"threads_started":1}
I20260809 20:17:29.966830 8880 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c25ff00fbf34fea8e145b671e99f521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c25ff00fbf34fea8e145b671e99f521" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 36557 } }
I20260809 20:17:29.967206 8880 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c25ff00fbf34fea8e145b671e99f521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:29.967330 8880 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c25ff00fbf34fea8e145b671e99f521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c25ff00fbf34fea8e145b671e99f521, State: Initialized, Role: FOLLOWER
I20260809 20:17:29.967818 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:17:29.967947 8880 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c25ff00fbf34fea8e145b671e99f521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c25ff00fbf34fea8e145b671e99f521" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 36557 } }
I20260809 20:17:29.968199 8880 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c25ff00fbf34fea8e145b671e99f521 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:17:29.968340 8880 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c25ff00fbf34fea8e145b671e99f521 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:17:29.968488 8880 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c25ff00fbf34fea8e145b671e99f521 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:29.971865 8880 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c25ff00fbf34fea8e145b671e99f521 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c25ff00fbf34fea8e145b671e99f521" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 36557 } }
I20260809 20:17:29.972442 8880 leader_election.cc:302] T 00000000000000000000000000000000 P 1c25ff00fbf34fea8e145b671e99f521 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c25ff00fbf34fea8e145b671e99f521; no voters:
I20260809 20:17:29.973127 8880 leader_election.cc:288] T 00000000000000000000000000000000 P 1c25ff00fbf34fea8e145b671e99f521 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:17:29.973207 8885 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c25ff00fbf34fea8e145b671e99f521 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:29.974730 8885 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c25ff00fbf34fea8e145b671e99f521 [term 1 LEADER]: Becoming Leader. State: Replica: 1c25ff00fbf34fea8e145b671e99f521, State: Running, Role: LEADER
I20260809 20:17:29.975402 8880 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c25ff00fbf34fea8e145b671e99f521 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:17:29.975286 8885 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c25ff00fbf34fea8e145b671e99f521 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c25ff00fbf34fea8e145b671e99f521" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 36557 } }
I20260809 20:17:29.977819 8888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c25ff00fbf34fea8e145b671e99f521 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c25ff00fbf34fea8e145b671e99f521. Latest consensus state: current_term: 1 leader_uuid: "1c25ff00fbf34fea8e145b671e99f521" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c25ff00fbf34fea8e145b671e99f521" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 36557 } } }
I20260809 20:17:29.978188 8888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c25ff00fbf34fea8e145b671e99f521 [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:29.978201 8886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c25ff00fbf34fea8e145b671e99f521 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c25ff00fbf34fea8e145b671e99f521" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c25ff00fbf34fea8e145b671e99f521" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 36557 } } }
I20260809 20:17:29.980249 8886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c25ff00fbf34fea8e145b671e99f521 [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:29.991492 8897 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:17:29.996829 8897 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:17:30.007035 8897 catalog_manager.cc:1347] Generated new cluster ID: ca663105c84b40b4a9bc3705d4c7af97
I20260809 20:17:30.007151 8897 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:17:30.023483 8897 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:17:30.025816 8897 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:17:30.033555 8883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:30.023774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45284 (local address 127.5.61.62:36557)
0809 20:17:30.023951 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:30.023959 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:30.024608 (+ 649us) server_negotiation.cc:407] Connection header received
0809 20:17:30.024810 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:30.024815 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:30.024898 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:30.025013 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:30.026347 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:30.027537 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:30.029399 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:30.030062 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:30.030202 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:30.031407 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:30.031414 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:30.031419 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:17:30.032503 (+ 1084us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:30.032604 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:17:30.032635 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:30.032770 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:30.032935 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 20:17:30.033029 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 20:17:30.033978 8897 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c25ff00fbf34fea8e145b671e99f521: Generated new TSK 0
I20260809 20:17:30.034782 8897 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:17:30.046962 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 8801
I20260809 20:17:30.048189 8873 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:30.139099 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 8801
2026-08-09T20:17:30Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1994 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:17:32 dist-test-slave-6k22 krb5kdc[8926](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:17:32 dist-test-slave-6k22 krb5kdc[8926](info): set up 2 sockets
Aug 09 20:17:32 dist-test-slave-6k22 krb5kdc[8926](info): commencing operation
krb5kdc: starting...
W20260809 20:17:33.153898 5364 mini_kdc.cc:121] Time spent starting KDC: real 2.971s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:17:33 dist-test-slave-6k22 krb5kdc[8926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:17:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:17:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:17:33.347682 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:39261
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:40691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:39261
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:33.669138 8941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:33.669447 8941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:33.669589 8941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:33.676867 8941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:17:33.676991 8941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:33.677095 8941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:33.677191 8941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:17:33.677273 8941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:17:33.687171 8941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:40691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:39261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:39261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:33.688678 8941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:33.689985 8941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:33.701407 8950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:33.701478 8946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:33.701498 8947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:33.702685 8941 server_base.cc:1034] running on GCE node
I20260809 20:17:33.703431 8941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:33.704931 8941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:33.706133 8941 hybrid_clock.cc:648] HybridClock initialized: now 1786306653706093 us; error 24 us; skew 500 ppm
I20260809 20:17:33.706475 8941 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:33 dist-test-slave-6k22 krb5kdc[8926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306653, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:33.716609 8941 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:17:33.718058 8941 webserver.cc:460] Webserver started at http://127.5.61.62:44599/ using document root <none> and password file <none>
I20260809 20:17:33.718731 8941 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:33.718840 8941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:33.719090 8941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:33.722016 8941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "56e05c145a234557af2debd48363a12b"
format_stamp: "Formatted at 2026-08-09 20:17:33 on dist-test-slave-6k22"
server_key: "6b10345b75b01460d450830e0aa8967c"
server_key_iv: "54fbf608bd9e3c3778b946278be64f11"
server_key_version: "encryptionkey@0"
I20260809 20:17:33.722704 8941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "56e05c145a234557af2debd48363a12b"
format_stamp: "Formatted at 2026-08-09 20:17:33 on dist-test-slave-6k22"
server_key: "6b10345b75b01460d450830e0aa8967c"
server_key_iv: "54fbf608bd9e3c3778b946278be64f11"
server_key_version: "encryptionkey@0"
I20260809 20:17:33.727804 8941 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.007s
I20260809 20:17:33.731036 8958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:33.732388 8941 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:17:33.732560 8941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "56e05c145a234557af2debd48363a12b"
format_stamp: "Formatted at 2026-08-09 20:17:33 on dist-test-slave-6k22"
server_key: "6b10345b75b01460d450830e0aa8967c"
server_key_iv: "54fbf608bd9e3c3778b946278be64f11"
server_key_version: "encryptionkey@0"
I20260809 20:17:33.732775 8941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:33.749563 8941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:33.753175 8941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:33.753449 8941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:33.771816 8941 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:39261
I20260809 20:17:33.771854 9014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:39261 every 8 connection(s)
I20260809 20:17:33.773075 8941 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:17:33.776321 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 8941
I20260809 20:17:33.776548 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:17:33.776880 9015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:33.777374 5364 external_mini_cluster.cc:1428] Setting key 413a1e715f9a3e4afe7aa9242082bc56
I20260809 20:17:33.791589 9015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56e05c145a234557af2debd48363a12b: Bootstrap starting.
Aug 09 20:17:33 dist-test-slave-6k22 krb5kdc[8926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:33.794591 9015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56e05c145a234557af2debd48363a12b: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:33.795675 9015 log.cc:824] T 00000000000000000000000000000000 P 56e05c145a234557af2debd48363a12b: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:33.798405 9015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56e05c145a234557af2debd48363a12b: No bootstrap required, opened a new log
I20260809 20:17:33.800290 9019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:33.779432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57506 (local address 127.5.61.62:39261)
0809 20:17:33.780356 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:33.780366 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:33.780408 (+ 42us) server_negotiation.cc:407] Connection header received
0809 20:17:33.782061 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:33.782080 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:33.782477 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:33.782804 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:33.784560 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:33.785815 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:33.787534 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:33.788279 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:33.788408 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:33.793083 (+ 4675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:33.793090 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:33.793106 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:33.793139 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:33.796342 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:33.797022 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:33.797033 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:33.797039 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:33.797214 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:33.797602 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:33.797609 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:33.797617 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:33.798095 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:33.798452 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:33.798863 (+ 411us) server_negotiation.cc:299] Negotiation successful
0809 20:17:33.799262 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":439,"threads_started":1}
I20260809 20:17:33.805871 9015 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56e05c145a234557af2debd48363a12b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e05c145a234557af2debd48363a12b" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39261 } }
I20260809 20:17:33.806237 9015 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56e05c145a234557af2debd48363a12b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:33.806341 9015 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56e05c145a234557af2debd48363a12b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e05c145a234557af2debd48363a12b, State: Initialized, Role: FOLLOWER
I20260809 20:17:33.806936 9015 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56e05c145a234557af2debd48363a12b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e05c145a234557af2debd48363a12b" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39261 } }
I20260809 20:17:33.807206 9015 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56e05c145a234557af2debd48363a12b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:17:33.807376 9015 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56e05c145a234557af2debd48363a12b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:17:33.807586 9015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56e05c145a234557af2debd48363a12b [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:33.810046 9015 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56e05c145a234557af2debd48363a12b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e05c145a234557af2debd48363a12b" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39261 } }
I20260809 20:17:33.810514 9015 leader_election.cc:302] T 00000000000000000000000000000000 P 56e05c145a234557af2debd48363a12b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56e05c145a234557af2debd48363a12b; no voters:
I20260809 20:17:33.811133 9015 leader_election.cc:288] T 00000000000000000000000000000000 P 56e05c145a234557af2debd48363a12b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:17:33.811254 9021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56e05c145a234557af2debd48363a12b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:33.812723 9021 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56e05c145a234557af2debd48363a12b [term 1 LEADER]: Becoming Leader. State: Replica: 56e05c145a234557af2debd48363a12b, State: Running, Role: LEADER
I20260809 20:17:33.813265 9015 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56e05c145a234557af2debd48363a12b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:17:33.813251 9021 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56e05c145a234557af2debd48363a12b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e05c145a234557af2debd48363a12b" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39261 } }
I20260809 20:17:33.816365 9023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56e05c145a234557af2debd48363a12b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56e05c145a234557af2debd48363a12b. Latest consensus state: current_term: 1 leader_uuid: "56e05c145a234557af2debd48363a12b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e05c145a234557af2debd48363a12b" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39261 } } }
I20260809 20:17:33.816932 9023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56e05c145a234557af2debd48363a12b [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:33.816882 9022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56e05c145a234557af2debd48363a12b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56e05c145a234557af2debd48363a12b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e05c145a234557af2debd48363a12b" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39261 } } }
I20260809 20:17:33.817178 9022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56e05c145a234557af2debd48363a12b [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:33.819681 9031 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:17:33.823727 9031 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:17:33.833005 9031 catalog_manager.cc:1347] Generated new cluster ID: 1903a70d641b4269aa593b15b30490f5
I20260809 20:17:33.833117 9031 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:17:33.839776 9031 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:17:33.841179 9031 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:17:33.854147 9031 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56e05c145a234557af2debd48363a12b: Generated new TSK 0
I20260809 20:17:33.854911 9031 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.61.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
I20260809 20:17:33.903476 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:39261
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:40691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:34.224530 9046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:34.224768 9046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:34.224907 9046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:34.231679 9046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:34.231779 9046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:34.231882 9046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:17:34.241075 9046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:40691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:39261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:34.242187 9046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:34.243384 9046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:34.255693 9057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:34.255750 9055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:34.256166 9053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:34.256585 9046 server_base.cc:1034] running on GCE node
I20260809 20:17:34.257107 9046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:34.257890 9046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:34.259078 9046 hybrid_clock.cc:648] HybridClock initialized: now 1786306654259047 us; error 26 us; skew 500 ppm
I20260809 20:17:34.259397 9046 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:34 dist-test-slave-6k22 krb5kdc[8926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306654, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:34.262444 9046 init.cc:377] Logged in from keytab as kudu/127.5.61.1@KRBTEST.COM (short username kudu)
I20260809 20:17:34.263846 9046 webserver.cc:460] Webserver started at http://127.5.61.1:43805/ using document root <none> and password file <none>
I20260809 20:17:34.264489 9046 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:34.264585 9046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:34.264875 9046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:34.267632 9046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "5a6ed8b21cc64f9cb55895a546869275"
format_stamp: "Formatted at 2026-08-09 20:17:34 on dist-test-slave-6k22"
server_key: "76b53342cd3678d534b89897494a6605"
server_key_iv: "56bc99540202e4af3d75d4bf96a36650"
server_key_version: "encryptionkey@0"
I20260809 20:17:34.268368 9046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "5a6ed8b21cc64f9cb55895a546869275"
format_stamp: "Formatted at 2026-08-09 20:17:34 on dist-test-slave-6k22"
server_key: "76b53342cd3678d534b89897494a6605"
server_key_iv: "56bc99540202e4af3d75d4bf96a36650"
server_key_version: "encryptionkey@0"
I20260809 20:17:34.273293 9046 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260809 20:17:34.276607 9064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:34.277912 9046 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 20:17:34.278053 9046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "5a6ed8b21cc64f9cb55895a546869275"
format_stamp: "Formatted at 2026-08-09 20:17:34 on dist-test-slave-6k22"
server_key: "76b53342cd3678d534b89897494a6605"
server_key_iv: "56bc99540202e4af3d75d4bf96a36650"
server_key_version: "encryptionkey@0"
I20260809 20:17:34.278215 9046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:34.305342 9046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:34.309194 9046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:34.309442 9046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:34.310316 9046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:34.311664 9046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:34.311741 9046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:34.311896 9046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:34.311959 9046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:34.345211 9046 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:33427
I20260809 20:17:34.345261 9185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:33427 every 8 connection(s)
I20260809 20:17:34.346434 9046 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:17:34.351229 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 9046
I20260809 20:17:34.351454 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:17:34.352133 5364 external_mini_cluster.cc:1428] Setting key 5c9f1968e71c52ff1e92b2bd63604c2f
Aug 09 20:17:34 dist-test-slave-6k22 krb5kdc[8926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306654, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:34.368013 9190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:34.349034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:50905 (local address 127.5.61.62:39261)
0809 20:17:34.349500 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:34.349506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:34.350366 (+ 860us) server_negotiation.cc:407] Connection header received
0809 20:17:34.352276 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:34.352282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:34.352381 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:34.352509 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:34.354203 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:34.354959 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:34.356726 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:34.357126 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:34.357265 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:34.361096 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:34.361103 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:34.361110 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:34.361155 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:34.364756 (+ 3601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:34.365444 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:34.365453 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:34.365458 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:34.365623 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:34.366026 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:34.366034 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:34.366040 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:34.366432 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:34.366606 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:34.367225 (+ 619us) server_negotiation.cc:299] Negotiation successful
0809 20:17:34.367489 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":222,"threads_started":1}
I20260809 20:17:34.368608 9193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:34.349530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:39261 (local address 127.5.61.1:50905)
0809 20:17:34.350206 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:34.350253 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:34.351992 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:34.352736 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:34.352743 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:34.353339 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:34.354048 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:34.354068 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:34.355112 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:34.355117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:34.356570 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:34.356599 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:34.357346 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:34.357353 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:34.357572 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:34.358263 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:34.358284 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:34.360869 (+ 2585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:34.364888 (+ 4019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:34.364895 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:34.364905 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:34.365312 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:34.365730 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:34.365733 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:34.365735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:34.365908 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:34.366557 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:34.366562 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:34.366836 (+ 274us) client_negotiation.cc:769] Sending connection context
0809 20:17:34.367062 (+ 226us) client_negotiation.cc:240] Negotiation successful
0809 20:17:34.367448 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":272,"threads_started":1}
I20260809 20:17:34.370226 9188 heartbeater.cc:347] Connected to a master server at 127.5.61.62:39261
I20260809 20:17:34.370558 9188 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:34.371518 9188 heartbeater.cc:511] Master 127.5.61.62:39261 requested a full tablet report, sending...
I20260809 20:17:34.373716 8976 ts_manager.cc:194] Registered new tserver with Master: 5a6ed8b21cc64f9cb55895a546869275 (127.5.61.1:33427)
WARNING: no policy specified for kudu/127.5.61.2@KRBTEST.COM; defaulting to no policy
I20260809 20:17:34.375262 8976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.1@KRBTEST.COM'} at 127.5.61.1:50905
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
I20260809 20:17:34.402226 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:39261
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:40691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:34.734831 9198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:34.735110 9198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:34.735272 9198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:34.742450 9198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:34.742549 9198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:34.742759 9198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:17:34.752246 9198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:40691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:39261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:34.753476 9198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:34.754760 9198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:34.767833 9203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:34.767910 9204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:34.767875 9206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:34.769079 9198 server_base.cc:1034] running on GCE node
I20260809 20:17:34.769789 9198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:34.770802 9198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:34.772039 9198 hybrid_clock.cc:648] HybridClock initialized: now 1786306654771969 us; error 75 us; skew 500 ppm
I20260809 20:17:34.772406 9198 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:34 dist-test-slave-6k22 krb5kdc[8926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306654, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:34.775895 9198 init.cc:377] Logged in from keytab as kudu/127.5.61.2@KRBTEST.COM (short username kudu)
I20260809 20:17:34.777511 9198 webserver.cc:460] Webserver started at http://127.5.61.2:44219/ using document root <none> and password file <none>
I20260809 20:17:34.778236 9198 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:34.778407 9198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:34.778752 9198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:34.782053 9198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "d885190df2034f6992f8604da3329e53"
format_stamp: "Formatted at 2026-08-09 20:17:34 on dist-test-slave-6k22"
server_key: "96f1b9053f3e68b825b2fa81951e4a2f"
server_key_iv: "a4619bfee96a422a1c75ab68e2567c69"
server_key_version: "encryptionkey@0"
I20260809 20:17:34.783000 9198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "d885190df2034f6992f8604da3329e53"
format_stamp: "Formatted at 2026-08-09 20:17:34 on dist-test-slave-6k22"
server_key: "96f1b9053f3e68b825b2fa81951e4a2f"
server_key_iv: "a4619bfee96a422a1c75ab68e2567c69"
server_key_version: "encryptionkey@0"
I20260809 20:17:34.788210 9198 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 20:17:34.791769 9213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:34.793051 9198 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:17:34.793205 9198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "d885190df2034f6992f8604da3329e53"
format_stamp: "Formatted at 2026-08-09 20:17:34 on dist-test-slave-6k22"
server_key: "96f1b9053f3e68b825b2fa81951e4a2f"
server_key_iv: "a4619bfee96a422a1c75ab68e2567c69"
server_key_version: "encryptionkey@0"
I20260809 20:17:34.793367 9198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:34.823633 9198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:34.828611 9198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:34.828905 9198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:34.829854 9198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:34.831291 9198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:34.831380 9198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:34.831482 9198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:34.831543 9198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:34.878692 9338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:46325 every 8 connection(s)
I20260809 20:17:34.878783 9198 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:46325
I20260809 20:17:34.881238 9198 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:17:34.882179 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 9198
I20260809 20:17:34.882381 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:17:34.882968 5364 external_mini_cluster.cc:1428] Setting key bcdb932f151442920f98d0abbf346005
WARNING: no policy specified for kudu/127.5.61.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.3@KRBTEST.COM" created.
Aug 09 20:17:34 dist-test-slave-6k22 krb5kdc[8926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306654, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:34.925053 9343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:34.885684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:40835 (local address 127.5.61.62:39261)
0809 20:17:34.900097 (+ 14413us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:34.900105 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:34.901152 (+ 1047us) server_negotiation.cc:407] Connection header received
0809 20:17:34.904041 (+ 2889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:34.904049 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:34.904141 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:34.904265 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:34.906606 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:34.907886 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:34.910590 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:34.911087 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:34.911186 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:34.916473 (+ 5287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:34.916482 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:34.916489 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:34.916567 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:34.920836 (+ 4269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:34.921858 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:34.921866 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:34.921871 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:34.922038 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:34.922608 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:34.922615 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:34.922620 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:34.923056 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:34.923221 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:34.924198 (+ 977us) server_negotiation.cc:299] Negotiation successful
0809 20:17:34.924477 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14252,"thread_start_us":14134,"threads_started":1}
I20260809 20:17:34.926402 9342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:34.885159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:39261 (local address 127.5.61.2:40835)
0809 20:17:34.900768 (+ 15609us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:34.900835 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:34.903713 (+ 2878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:34.904627 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:34.904643 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:34.905507 (+ 864us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:34.906374 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:34.906403 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:34.908094 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:34.908102 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:34.910439 (+ 2337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:34.910457 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:34.911326 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:34.911337 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:34.911734 (+ 397us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:34.912602 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:34.912639 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:34.916154 (+ 3515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:34.921015 (+ 4861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:34.921027 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:34.921045 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:34.921726 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:34.922164 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:34.922172 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:34.922178 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:34.922441 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:34.923215 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:34.923226 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:34.923745 (+ 519us) client_negotiation.cc:769] Sending connection context
0809 20:17:34.924090 (+ 345us) client_negotiation.cc:240] Negotiation successful
0809 20:17:34.924691 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15225,"thread_start_us":15018,"threads_started":1}
I20260809 20:17:34.928296 9340 heartbeater.cc:347] Connected to a master server at 127.5.61.62:39261
I20260809 20:17:34.928635 9340 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:34.929497 9340 heartbeater.cc:511] Master 127.5.61.62:39261 requested a full tablet report, sending...
I20260809 20:17:34.931058 8976 ts_manager.cc:194] Registered new tserver with Master: d885190df2034f6992f8604da3329e53 (127.5.61.2:46325)
I20260809 20:17:34.931942 8976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.2@KRBTEST.COM'} at 127.5.61.2:40835
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
I20260809 20:17:34.939985 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:39261
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:40691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:35.265765 9346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:35.266063 9346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:35.266249 9346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:35.273394 9346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:35.273533 9346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:35.273720 9346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:17:35.283393 9346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:40691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:39261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:35.284651 9346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:35.285961 9346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:35.298872 9352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:35.299064 9353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:35.299077 9355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:35.300305 9346 server_base.cc:1034] running on GCE node
I20260809 20:17:35.300894 9346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:35.301760 9346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:35.303012 9346 hybrid_clock.cc:648] HybridClock initialized: now 1786306655302968 us; error 50 us; skew 500 ppm
I20260809 20:17:35.303329 9346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:35 dist-test-slave-6k22 krb5kdc[8926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306655, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:35.306565 9346 init.cc:377] Logged in from keytab as kudu/127.5.61.3@KRBTEST.COM (short username kudu)
I20260809 20:17:35.308015 9346 webserver.cc:460] Webserver started at http://127.5.61.3:34159/ using document root <none> and password file <none>
I20260809 20:17:35.308619 9346 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:35.308734 9346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:35.308990 9346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:35.311895 9346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "dd7d6bd573e14735a9f5fbf8062b012f"
format_stamp: "Formatted at 2026-08-09 20:17:35 on dist-test-slave-6k22"
server_key: "68ad10f6898847da291846a0399b4c76"
server_key_iv: "9ac7b872fbb3f9874111312ad59a809c"
server_key_version: "encryptionkey@0"
I20260809 20:17:35.312608 9346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "dd7d6bd573e14735a9f5fbf8062b012f"
format_stamp: "Formatted at 2026-08-09 20:17:35 on dist-test-slave-6k22"
server_key: "68ad10f6898847da291846a0399b4c76"
server_key_iv: "9ac7b872fbb3f9874111312ad59a809c"
server_key_version: "encryptionkey@0"
I20260809 20:17:35.317951 9346 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 20:17:35.321211 9363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:35.322500 9346 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 20:17:35.322628 9346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "dd7d6bd573e14735a9f5fbf8062b012f"
format_stamp: "Formatted at 2026-08-09 20:17:35 on dist-test-slave-6k22"
server_key: "68ad10f6898847da291846a0399b4c76"
server_key_iv: "9ac7b872fbb3f9874111312ad59a809c"
server_key_version: "encryptionkey@0"
I20260809 20:17:35.322775 9346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:35.344148 9346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:35.348438 9346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:35.348729 9346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:35.349597 9346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:35.350826 9346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:35.350937 9346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:35.351082 9346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:35.351161 9346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:35.378312 9188 heartbeater.cc:503] Master 127.5.61.62:39261 was elected leader, sending a full tablet report...
I20260809 20:17:35.386127 9346 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:40369
I20260809 20:17:35.386185 9499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:40369 every 8 connection(s)
I20260809 20:17:35.387462 9346 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:17:35.388669 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 9346
I20260809 20:17:35.388885 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:17:35.389506 5364 external_mini_cluster.cc:1428] Setting key 42873adca3a26df003326c8a13b1665c
Aug 09 20:17:35 dist-test-slave-6k22 krb5kdc[8926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306655, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:35.407898 9343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:35.390454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:45033 (local address 127.5.61.62:39261)
0809 20:17:35.390660 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:35.390666 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:35.392125 (+ 1459us) server_negotiation.cc:407] Connection header received
0809 20:17:35.394069 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:35.394073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:35.394137 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:35.394220 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:35.395689 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:35.396419 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:35.397938 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:35.398515 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:35.398613 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:35.402032 (+ 3419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:35.402037 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:35.402040 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:35.402070 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:35.405322 (+ 3252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:35.405977 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:35.405981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:35.405983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:35.406091 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:35.406451 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:35.406454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:35.406456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:35.406709 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:35.406817 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:35.407438 (+ 621us) server_negotiation.cc:299] Negotiation successful
0809 20:17:35.407671 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 20:17:35.408841 9502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:35.390948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:39261 (local address 127.5.61.3:45033)
0809 20:17:35.391951 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:35.392001 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:35.393818 (+ 1817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:35.394429 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:35.394436 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:35.394939 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:35.395483 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:35.395500 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:35.396548 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:35.396552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:35.397787 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:35.397799 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:35.398700 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:35.398704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:35.398908 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:35.399684 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:35.399732 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:35.401836 (+ 2104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:35.405444 (+ 3608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:35.405450 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:35.405461 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:35.405872 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:35.406178 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:35.406181 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:35.406183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:35.406351 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:35.406812 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:35.406817 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:35.407117 (+ 300us) client_negotiation.cc:769] Sending connection context
0809 20:17:35.407339 (+ 222us) client_negotiation.cc:240] Negotiation successful
0809 20:17:35.407784 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":383,"threads_started":1}
I20260809 20:17:35.410158 9500 heartbeater.cc:347] Connected to a master server at 127.5.61.62:39261
I20260809 20:17:35.410465 9500 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:35.411175 9500 heartbeater.cc:511] Master 127.5.61.62:39261 requested a full tablet report, sending...
I20260809 20:17:35.412616 8976 ts_manager.cc:194] Registered new tserver with Master: dd7d6bd573e14735a9f5fbf8062b012f (127.5.61.3:40369)
I20260809 20:17:35.413363 8976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.3@KRBTEST.COM'} at 127.5.61.3:45033
I20260809 20:17:35.417447 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 20:17:35 dist-test-slave-6k22 krb5kdc[8926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306655, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 20:17:35 dist-test-slave-6k22 krb5kdc[8926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306655, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:35.467317 9343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:35.453990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57508 (local address 127.5.61.62:39261)
0809 20:17:35.454232 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:35.454253 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:35.454663 (+ 410us) server_negotiation.cc:407] Connection header received
0809 20:17:35.454809 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:35.454813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:35.454878 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:35.454984 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:35.456355 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:35.457091 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:35.458262 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:35.458674 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:35.458769 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:35.462515 (+ 3746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:35.462520 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:35.462523 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:35.462553 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:35.464965 (+ 2412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:35.465533 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:35.465537 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:35.465539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:35.465657 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:35.466012 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:35.466016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:35.466017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:35.466369 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:35.466573 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:35.466788 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 20:17:35.466939 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 20:17:35.467935 8976 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:57508
I20260809 20:17:35.469545 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 9046
Aug 09 20:17:35 dist-test-slave-6k22 krb5kdc[8926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306655, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.61.1@KRBTEST.COM
I20260809 20:17:35.485962 9513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:35.470665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35428 (local address 127.5.61.1:33427)
0809 20:17:35.471275 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:35.471282 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:35.471304 (+ 22us) server_negotiation.cc:407] Connection header received
0809 20:17:35.471357 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:35.471365 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:35.471638 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:35.471818 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:35.472943 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:35.473990 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:35.475263 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:35.475920 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:35.476058 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:35.480303 (+ 4245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:35.480315 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:35.480329 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:35.480368 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:35.483490 (+ 3122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:35.484163 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:35.484169 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:35.484171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:35.484304 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:35.484628 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:35.484630 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:35.484632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:35.485006 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:35.485216 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:35.485467 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 20:17:35.485662 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":326,"threads_started":1}
W20260809 20:17:35.487226 9180 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:35428
W20260809 20:17:35.488379 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 9046: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 20:17:35.488591 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 9046
I20260809 20:17:35.506789 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 9198
Aug 09 20:17:35 dist-test-slave-6k22 krb5kdc[8926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306655, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.61.2@KRBTEST.COM
I20260809 20:17:35.523027 9514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:35.507677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33550 (local address 127.5.61.2:46325)
0809 20:17:35.508178 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:35.508186 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:35.508230 (+ 44us) server_negotiation.cc:407] Connection header received
0809 20:17:35.508286 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:35.508292 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:35.508497 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:35.508670 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:35.509934 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:35.510926 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:35.512327 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:35.512901 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:35.513043 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:35.517322 (+ 4279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:35.517332 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:35.517346 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:35.517399 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:35.520162 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:35.520806 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:35.520813 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:35.520815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:35.520950 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:35.521347 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:35.521351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:35.521353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:35.521688 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:35.522023 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:35.522395 (+ 372us) server_negotiation.cc:299] Negotiation successful
0809 20:17:35.522666 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":264,"threads_started":1}
W20260809 20:17:35.524503 9334 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33550
W20260809 20:17:35.525642 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 9198: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 20:17:35.525770 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 9198
I20260809 20:17:35.546729 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 9346
Aug 09 20:17:35 dist-test-slave-6k22 krb5kdc[8926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306655, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.61.3@KRBTEST.COM
I20260809 20:17:35.563131 9515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:35.547777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38186 (local address 127.5.61.3:40369)
0809 20:17:35.548401 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:35.548408 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:35.548435 (+ 27us) server_negotiation.cc:407] Connection header received
0809 20:17:35.548490 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:35.548496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:35.548692 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:35.548862 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:35.550288 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:35.551283 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:35.552608 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:35.553157 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:35.553318 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:35.557559 (+ 4241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:35.557570 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:35.557583 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:35.557617 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:35.560356 (+ 2739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:35.561011 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:35.561017 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:35.561019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:35.561138 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:35.561537 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:35.561541 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:35.561543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:35.561866 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:35.562209 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:35.562513 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 20:17:35.562779 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":326,"threads_started":1}
W20260809 20:17:35.564497 9495 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38186
W20260809 20:17:35.565567 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 9346: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 20:17:35.565768 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 9346
I20260809 20:17:35.588332 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 8941
I20260809 20:17:35.589545 9009 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:35.692889 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 8941
2026-08-09T20:17:35Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (5556 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:17:35 dist-test-slave-6k22 krb5kdc[9519](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:17:35 dist-test-slave-6k22 krb5kdc[9519](info): set up 2 sockets
Aug 09 20:17:35 dist-test-slave-6k22 krb5kdc[9519](info): commencing operation
krb5kdc: starting...
W20260809 20:17:36.850380 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.002s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:17:36 dist-test-slave-6k22 krb5kdc[9519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:17:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:17:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:17:37.056764 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:39551
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:39551
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:37.371960 9535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:37.372232 9535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:37.372442 9535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:37.379376 9535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:17:37.379470 9535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:37.379602 9535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:37.379739 9535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:17:37.379835 9535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:17:37.389115 9535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:39551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:39551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:37.390405 9535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:37.391731 9535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:37.403788 9540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:37.403838 9541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:37.403992 9544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:37.404508 9535 server_base.cc:1034] running on GCE node
I20260809 20:17:37.405279 9535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:37.406615 9535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:37.407807 9535 hybrid_clock.cc:648] HybridClock initialized: now 1786306657407753 us; error 54 us; skew 500 ppm
I20260809 20:17:37.408142 9535 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:37 dist-test-slave-6k22 krb5kdc[9519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306657, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:37.411161 9535 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:17:37.412750 9535 webserver.cc:460] Webserver started at http://127.5.61.62:41025/ using document root <none> and password file <none>
I20260809 20:17:37.413492 9535 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:37.413623 9535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:37.413972 9535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:37.416952 9535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "1847e57b533a47c8a2df4525860089e9"
format_stamp: "Formatted at 2026-08-09 20:17:37 on dist-test-slave-6k22"
server_key: "5b2b599a9c656c48d73a0cf63dca4c94"
server_key_iv: "60db08687803c2e97da27f2084eab73b"
server_key_version: "encryptionkey@0"
I20260809 20:17:37.417764 9535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "1847e57b533a47c8a2df4525860089e9"
format_stamp: "Formatted at 2026-08-09 20:17:37 on dist-test-slave-6k22"
server_key: "5b2b599a9c656c48d73a0cf63dca4c94"
server_key_iv: "60db08687803c2e97da27f2084eab73b"
server_key_version: "encryptionkey@0"
I20260809 20:17:37.423413 9535 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 20:17:37.426785 9551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:37.428179 9535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:17:37.428387 9535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "1847e57b533a47c8a2df4525860089e9"
format_stamp: "Formatted at 2026-08-09 20:17:37 on dist-test-slave-6k22"
server_key: "5b2b599a9c656c48d73a0cf63dca4c94"
server_key_iv: "60db08687803c2e97da27f2084eab73b"
server_key_version: "encryptionkey@0"
I20260809 20:17:37.428622 9535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:37.455775 9535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:37.459450 9535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:37.459769 9535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:37.478192 9535 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:39551
I20260809 20:17:37.478227 9615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:39551 every 8 connection(s)
I20260809 20:17:37.479627 9535 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:17:37.483161 9616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:37.484802 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 9535
I20260809 20:17:37.485064 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:17:37.485898 5364 external_mini_cluster.cc:1428] Setting key 710173b0b64f4662fd1026dc17e066be
I20260809 20:17:37.497800 9616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1847e57b533a47c8a2df4525860089e9: Bootstrap starting.
Aug 09 20:17:37 dist-test-slave-6k22 krb5kdc[9519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:37.501438 9616 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1847e57b533a47c8a2df4525860089e9: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:37.502619 9616 log.cc:824] T 00000000000000000000000000000000 P 1847e57b533a47c8a2df4525860089e9: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:37.505664 9616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1847e57b533a47c8a2df4525860089e9: No bootstrap required, opened a new log
I20260809 20:17:37.508917 9619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:37.488024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47634 (local address 127.5.61.62:39551)
0809 20:17:37.488827 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:37.488842 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:37.488883 (+ 41us) server_negotiation.cc:407] Connection header received
0809 20:17:37.490402 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:37.490421 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:37.490817 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:37.491101 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:37.492977 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:37.494297 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:37.496154 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:37.496778 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:37.496901 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:37.501366 (+ 4465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:37.501376 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:37.501388 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:37.501423 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:37.504924 (+ 3501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:37.505603 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:37.505613 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:37.505624 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:37.505890 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:37.506287 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:37.506292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:37.506297 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:37.506751 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:37.507071 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:37.507527 (+ 456us) server_negotiation.cc:299] Negotiation successful
0809 20:17:37.507936 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":354,"threads_started":1}
I20260809 20:17:37.513118 9616 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1847e57b533a47c8a2df4525860089e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1847e57b533a47c8a2df4525860089e9" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39551 } }
I20260809 20:17:37.513466 9616 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1847e57b533a47c8a2df4525860089e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:37.513576 9616 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1847e57b533a47c8a2df4525860089e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1847e57b533a47c8a2df4525860089e9, State: Initialized, Role: FOLLOWER
I20260809 20:17:37.514183 9616 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1847e57b533a47c8a2df4525860089e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1847e57b533a47c8a2df4525860089e9" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39551 } }
I20260809 20:17:37.514437 9616 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1847e57b533a47c8a2df4525860089e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:17:37.514575 9616 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1847e57b533a47c8a2df4525860089e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:17:37.514752 9616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1847e57b533a47c8a2df4525860089e9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:37.517175 9616 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1847e57b533a47c8a2df4525860089e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1847e57b533a47c8a2df4525860089e9" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39551 } }
I20260809 20:17:37.517652 9616 leader_election.cc:302] T 00000000000000000000000000000000 P 1847e57b533a47c8a2df4525860089e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1847e57b533a47c8a2df4525860089e9; no voters:
I20260809 20:17:37.518199 9616 leader_election.cc:288] T 00000000000000000000000000000000 P 1847e57b533a47c8a2df4525860089e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:17:37.518337 9621 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1847e57b533a47c8a2df4525860089e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:37.519114 9621 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1847e57b533a47c8a2df4525860089e9 [term 1 LEADER]: Becoming Leader. State: Replica: 1847e57b533a47c8a2df4525860089e9, State: Running, Role: LEADER
I20260809 20:17:37.519711 9621 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1847e57b533a47c8a2df4525860089e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1847e57b533a47c8a2df4525860089e9" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39551 } }
I20260809 20:17:37.520470 9616 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1847e57b533a47c8a2df4525860089e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:17:37.522925 9622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1847e57b533a47c8a2df4525860089e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1847e57b533a47c8a2df4525860089e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1847e57b533a47c8a2df4525860089e9" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39551 } } }
I20260809 20:17:37.522913 9623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1847e57b533a47c8a2df4525860089e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1847e57b533a47c8a2df4525860089e9. Latest consensus state: current_term: 1 leader_uuid: "1847e57b533a47c8a2df4525860089e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1847e57b533a47c8a2df4525860089e9" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39551 } } }
I20260809 20:17:37.523407 9623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1847e57b533a47c8a2df4525860089e9 [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:37.523404 9622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1847e57b533a47c8a2df4525860089e9 [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:37.528206 9628 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:17:37.532640 9628 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:17:37.540288 9628 catalog_manager.cc:1347] Generated new cluster ID: 08c82d2f468f4910883bee3ae5e63d4e
I20260809 20:17:37.540395 9628 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:17:37.556710 9628 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:17:37.557992 9628 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:17:37.569401 9628 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1847e57b533a47c8a2df4525860089e9: Generated new TSK 0
I20260809 20:17:37.570317 9628 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.61.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
I20260809 20:17:37.630775 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:39551
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:37.970590 9645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:37.970875 9645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:37.970974 9645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:37.978175 9645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:37.978292 9645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:37.978528 9645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:17:37.988969 9645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:39551
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:37.990703 9645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:37.992925 9645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:38.007848 9651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:38.007874 9653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:38.008442 9650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:38.009032 9645 server_base.cc:1034] running on GCE node
I20260809 20:17:38.009590 9645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:38.010651 9645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:38.011871 9645 hybrid_clock.cc:648] HybridClock initialized: now 1786306658011819 us; error 46 us; skew 500 ppm
I20260809 20:17:38.012133 9645 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:38 dist-test-slave-6k22 krb5kdc[9519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306658, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:38.015388 9645 init.cc:377] Logged in from keytab as kudu/127.5.61.1@KRBTEST.COM (short username kudu)
I20260809 20:17:38.016870 9645 webserver.cc:460] Webserver started at http://127.5.61.1:45951/ using document root <none> and password file <none>
I20260809 20:17:38.017517 9645 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:38.017649 9645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:38.017975 9645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:38.021096 9645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "cf8424bd316c44f2ad1cf72fc40ad6ef"
format_stamp: "Formatted at 2026-08-09 20:17:38 on dist-test-slave-6k22"
server_key: "e6fd1ac1b65a725e0dfd6374733cc0c8"
server_key_iv: "ff5e88985b4e64065cb2f6d685a29f8d"
server_key_version: "encryptionkey@0"
I20260809 20:17:38.021970 9645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "cf8424bd316c44f2ad1cf72fc40ad6ef"
format_stamp: "Formatted at 2026-08-09 20:17:38 on dist-test-slave-6k22"
server_key: "e6fd1ac1b65a725e0dfd6374733cc0c8"
server_key_iv: "ff5e88985b4e64065cb2f6d685a29f8d"
server_key_version: "encryptionkey@0"
I20260809 20:17:38.027200 9645 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 20:17:38.030565 9660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:38.031944 9645 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:17:38.032104 9645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "cf8424bd316c44f2ad1cf72fc40ad6ef"
format_stamp: "Formatted at 2026-08-09 20:17:38 on dist-test-slave-6k22"
server_key: "e6fd1ac1b65a725e0dfd6374733cc0c8"
server_key_iv: "ff5e88985b4e64065cb2f6d685a29f8d"
server_key_version: "encryptionkey@0"
I20260809 20:17:38.032302 9645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:38.047168 9645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:38.051296 9645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:38.051605 9645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:38.052484 9645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:38.053848 9645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:38.053951 9645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:38.054109 9645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:38.054195 9645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:38.087035 9645 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:35105
I20260809 20:17:38.087082 9792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:35105 every 8 connection(s)
I20260809 20:17:38.088332 9645 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:17:38.091696 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 9645
I20260809 20:17:38.091919 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:17:38.092619 5364 external_mini_cluster.cc:1428] Setting key ccd730eb9c70587427d7495e5916eae2
Aug 09 20:17:38 dist-test-slave-6k22 krb5kdc[9519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306658, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:38.107667 9797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:38.090902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:44905 (local address 127.5.61.62:39551)
0809 20:17:38.091369 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:38.091376 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:38.092439 (+ 1063us) server_negotiation.cc:407] Connection header received
0809 20:17:38.094287 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:38.094293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:38.094374 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:38.094487 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:38.096073 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:38.096828 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:38.098313 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:38.098777 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:38.098915 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:38.102225 (+ 3310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:38.102231 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:38.102240 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:38.102271 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:38.104694 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:38.105415 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:38.105425 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:38.105427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:38.105547 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:38.105999 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:38.106003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:38.106005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:38.106262 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:38.106420 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:38.107095 (+ 675us) server_negotiation.cc:299] Negotiation successful
0809 20:17:38.107329 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":199,"threads_started":1}
I20260809 20:17:38.108510 9798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:38.091423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:39551 (local address 127.5.61.1:44905)
0809 20:17:38.092262 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:38.092315 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:38.094014 (+ 1699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:38.094713 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:38.094721 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:38.095227 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:38.095903 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:38.095921 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:38.097002 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:38.097006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:38.098156 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:38.098167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:38.099012 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:38.099022 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:38.099250 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:38.099952 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:38.099972 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:38.102009 (+ 2037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:38.104853 (+ 2844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:38.104862 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:38.104875 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:38.105283 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:38.105662 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:38.105667 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:38.105671 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:38.105869 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:38.106393 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:38.106400 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:38.106703 (+ 303us) client_negotiation.cc:769] Sending connection context
0809 20:17:38.106976 (+ 273us) client_negotiation.cc:240] Negotiation successful
0809 20:17:38.107438 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":394,"threads_started":1}
I20260809 20:17:38.110046 9794 heartbeater.cc:347] Connected to a master server at 127.5.61.62:39551
I20260809 20:17:38.110312 9794 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:38.110977 9794 heartbeater.cc:511] Master 127.5.61.62:39551 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.5.61.2@KRBTEST.COM; defaulting to no policy
I20260809 20:17:38.113029 9570 ts_manager.cc:194] Registered new tserver with Master: cf8424bd316c44f2ad1cf72fc40ad6ef (127.5.61.1:35105)
I20260809 20:17:38.114400 9570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.1@KRBTEST.COM'} at 127.5.61.1:44905
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
I20260809 20:17:38.138777 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:39551
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:38.475402 9801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:38.475776 9801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:38.475983 9801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:38.483341 9801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:38.483517 9801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:38.483783 9801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:17:38.493790 9801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:39551
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:38.495074 9801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:38.496615 9801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:38.509953 9806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:38.509902 9809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:38.509981 9807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:38.510972 9801 server_base.cc:1034] running on GCE node
I20260809 20:17:38.511499 9801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:38.512470 9801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:38.513677 9801 hybrid_clock.cc:648] HybridClock initialized: now 1786306658513610 us; error 58 us; skew 500 ppm
I20260809 20:17:38.513993 9801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:38 dist-test-slave-6k22 krb5kdc[9519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306658, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:38.517307 9801 init.cc:377] Logged in from keytab as kudu/127.5.61.2@KRBTEST.COM (short username kudu)
I20260809 20:17:38.518821 9801 webserver.cc:460] Webserver started at http://127.5.61.2:35931/ using document root <none> and password file <none>
I20260809 20:17:38.519526 9801 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:38.519747 9801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:38.520174 9801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:38.523427 9801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "b29f52253af442b5a9a097ca9b09a6f5"
format_stamp: "Formatted at 2026-08-09 20:17:38 on dist-test-slave-6k22"
server_key: "c2e738b7849dceeacf5af9a2336d84ef"
server_key_iv: "2313168eab564fcd3116b972fcb299cd"
server_key_version: "encryptionkey@0"
I20260809 20:17:38.524324 9801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "b29f52253af442b5a9a097ca9b09a6f5"
format_stamp: "Formatted at 2026-08-09 20:17:38 on dist-test-slave-6k22"
server_key: "c2e738b7849dceeacf5af9a2336d84ef"
server_key_iv: "2313168eab564fcd3116b972fcb299cd"
server_key_version: "encryptionkey@0"
I20260809 20:17:38.529831 9801 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 20:17:38.533492 9817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:38.534762 9801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 20:17:38.534940 9801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "b29f52253af442b5a9a097ca9b09a6f5"
format_stamp: "Formatted at 2026-08-09 20:17:38 on dist-test-slave-6k22"
server_key: "c2e738b7849dceeacf5af9a2336d84ef"
server_key_iv: "2313168eab564fcd3116b972fcb299cd"
server_key_version: "encryptionkey@0"
I20260809 20:17:38.535159 9801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:38.558997 9801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:38.563544 9801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:38.563900 9801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:38.564941 9801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:38.566363 9801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:38.566474 9801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:38.566691 9801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:38.566756 9801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:38.604557 9801 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:45671
I20260809 20:17:38.604637 9955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:45671 every 8 connection(s)
I20260809 20:17:38.605930 9801 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:17:38.607877 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 9801
I20260809 20:17:38.608104 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:17:38.608664 5364 external_mini_cluster.cc:1428] Setting key e8cd129daeb7e4c0e570d3881947aec5
Aug 09 20:17:38 dist-test-slave-6k22 krb5kdc[9519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306658, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:38.628890 9958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:38.608814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:48995 (local address 127.5.61.62:39551)
0809 20:17:38.609339 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:38.609346 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:38.610398 (+ 1052us) server_negotiation.cc:407] Connection header received
0809 20:17:38.612333 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:38.612343 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:38.612439 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:38.612567 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:38.614057 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:38.615289 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:38.616844 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:38.617594 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:38.617764 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:38.622409 (+ 4645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:38.622415 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:38.622418 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:38.622449 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:38.625205 (+ 2756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:38.626218 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:38.626222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:38.626225 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:38.626536 (+ 311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:38.627040 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:38.627044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:38.627046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:38.627369 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:38.627489 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:38.628450 (+ 961us) server_negotiation.cc:299] Negotiation successful
0809 20:17:38.628630 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":270,"threads_started":1}
I20260809 20:17:38.630402 9959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:38.609376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:39551 (local address 127.5.61.2:48995)
0809 20:17:38.610230 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:38.610281 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:38.612061 (+ 1780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:38.612778 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:38.612786 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:38.613271 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:38.613865 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:38.613882 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:38.615431 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:38.615434 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:38.616686 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:38.616698 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:38.617829 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:38.617838 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:38.618134 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:38.618954 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:38.618989 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:38.622153 (+ 3164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:38.625370 (+ 3217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:38.625382 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:38.625400 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:38.626098 (+ 698us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:38.626652 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:38.626659 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:38.626665 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:38.626925 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:38.627527 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:38.627538 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:38.628026 (+ 488us) client_negotiation.cc:769] Sending connection context
0809 20:17:38.628358 (+ 332us) client_negotiation.cc:240] Negotiation successful
0809 20:17:38.628928 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":380,"threads_started":1}
WARNING: no policy specified for kudu/127.5.61.3@KRBTEST.COM; defaulting to no policy
I20260809 20:17:38.631923 9956 heartbeater.cc:347] Connected to a master server at 127.5.61.62:39551
I20260809 20:17:38.632298 9956 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:38.633033 9956 heartbeater.cc:511] Master 127.5.61.62:39551 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.3@KRBTEST.COM" created.
I20260809 20:17:38.634616 9570 ts_manager.cc:194] Registered new tserver with Master: b29f52253af442b5a9a097ca9b09a6f5 (127.5.61.2:45671)
I20260809 20:17:38.635444 9570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.2@KRBTEST.COM'} at 127.5.61.2:48995
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
I20260809 20:17:38.659137 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:39551
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:38.982684 9962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:38.982983 9962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:38.983081 9962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:38.990106 9962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:38.990206 9962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:38.990314 9962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:17:38.999904 9962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:39551
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:39.001116 9962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:39.002344 9962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:39.015507 9969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:39.015529 9972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:39.015501 9970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:39.016404 9962 server_base.cc:1034] running on GCE node
I20260809 20:17:39.016992 9962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:39.018028 9962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:39.019263 9962 hybrid_clock.cc:648] HybridClock initialized: now 1786306659019221 us; error 32 us; skew 500 ppm
I20260809 20:17:39.019629 9962 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:39 dist-test-slave-6k22 krb5kdc[9519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306659, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:39.029141 9962 init.cc:377] Logged in from keytab as kudu/127.5.61.3@KRBTEST.COM (short username kudu)
I20260809 20:17:39.030653 9962 webserver.cc:460] Webserver started at http://127.5.61.3:41483/ using document root <none> and password file <none>
I20260809 20:17:39.031320 9962 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:39.031466 9962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:39.031819 9962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:39.034718 9962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "9e8f2fe1b986466286197fe4af333cd6"
format_stamp: "Formatted at 2026-08-09 20:17:39 on dist-test-slave-6k22"
server_key: "9b218c339eee5b3cf10d7e741ea23c8c"
server_key_iv: "95dc3e5ae96c62a38fe9626d81a49aa3"
server_key_version: "encryptionkey@0"
I20260809 20:17:39.035565 9962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "9e8f2fe1b986466286197fe4af333cd6"
format_stamp: "Formatted at 2026-08-09 20:17:39 on dist-test-slave-6k22"
server_key: "9b218c339eee5b3cf10d7e741ea23c8c"
server_key_iv: "95dc3e5ae96c62a38fe9626d81a49aa3"
server_key_version: "encryptionkey@0"
I20260809 20:17:39.040863 9962 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 20:17:39.044472 9979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:39.045754 9962 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:17:39.045929 9962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "9e8f2fe1b986466286197fe4af333cd6"
format_stamp: "Formatted at 2026-08-09 20:17:39 on dist-test-slave-6k22"
server_key: "9b218c339eee5b3cf10d7e741ea23c8c"
server_key_iv: "95dc3e5ae96c62a38fe9626d81a49aa3"
server_key_version: "encryptionkey@0"
I20260809 20:17:39.046131 9962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:39.074986 9962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:39.079134 9962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:39.079437 9962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:39.080394 9962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:39.081681 9962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:39.081812 9962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:39.081959 9962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:39.082041 9962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:39.118037 9794 heartbeater.cc:503] Master 127.5.61.62:39551 was elected leader, sending a full tablet report...
I20260809 20:17:39.123401 9962 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:33279
I20260809 20:17:39.123468 10110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:33279 every 8 connection(s)
I20260809 20:17:39.125013 9962 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:17:39.127022 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 9962
I20260809 20:17:39.127265 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:17:39.127879 5364 external_mini_cluster.cc:1428] Setting key b10ba619b4c47116db27545e348816a6
Aug 09 20:17:39 dist-test-slave-6k22 krb5kdc[9519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306659, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:39.152047 10115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:39.129311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:39327 (local address 127.5.61.62:39551)
0809 20:17:39.129861 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:39.129867 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:39.131021 (+ 1154us) server_negotiation.cc:407] Connection header received
0809 20:17:39.133395 (+ 2374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:39.133399 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:39.133473 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:39.133584 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:39.135421 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:39.136387 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:39.139009 (+ 2622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:39.139629 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:39.139808 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:39.144225 (+ 4417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:39.144232 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:39.144236 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:39.144277 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:39.148218 (+ 3941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:39.149145 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:39.149152 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:39.149158 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:39.149346 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:39.149943 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:39.149951 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:39.149954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:39.150409 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:39.150569 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:39.151383 (+ 814us) server_negotiation.cc:299] Negotiation successful
0809 20:17:39.151646 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":311,"threads_started":1}
I20260809 20:17:39.153088 10116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:39.129982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:39551 (local address 127.5.61.3:39327)
0809 20:17:39.130842 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:39.130902 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:39.133090 (+ 2188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:39.133855 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:39.133882 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:39.134662 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:39.135240 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:39.135258 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:39.136593 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:39.136600 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:39.138862 (+ 2262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:39.138877 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:39.140074 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:39.140080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:39.140287 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:39.140995 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:39.141017 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:39.143983 (+ 2966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:39.148352 (+ 4369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:39.148369 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:39.148386 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:39.149007 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:39.149490 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:39.149495 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:39.149498 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:39.149802 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:39.150564 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:39.150575 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:39.150985 (+ 410us) client_negotiation.cc:769] Sending connection context
0809 20:17:39.151290 (+ 305us) client_negotiation.cc:240] Negotiation successful
0809 20:17:39.151849 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":414,"threads_started":1}
I20260809 20:17:39.154686 10111 heartbeater.cc:347] Connected to a master server at 127.5.61.62:39551
I20260809 20:17:39.155016 10111 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:39.155992 10111 heartbeater.cc:511] Master 127.5.61.62:39551 requested a full tablet report, sending...
I20260809 20:17:39.157784 9570 ts_manager.cc:194] Registered new tserver with Master: 9e8f2fe1b986466286197fe4af333cd6 (127.5.61.3:33279)
I20260809 20:17:39.158553 9570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.3@KRBTEST.COM'} at 127.5.61.3:39327
I20260809 20:17:39.169397 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 20:17:39 dist-test-slave-6k22 krb5kdc[9519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 20:17:39 dist-test-slave-6k22 krb5kdc[9519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.1@KRBTEST.COM
I20260809 20:17:39.220366 10121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:39.205852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38714 (local address 127.5.61.1:35105)
0809 20:17:39.206515 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:39.206523 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:39.206548 (+ 25us) server_negotiation.cc:407] Connection header received
0809 20:17:39.206612 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:39.206618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:39.206836 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:39.206966 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:39.208340 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:39.209303 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:39.210520 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:39.211066 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:39.211197 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:39.215068 (+ 3871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:39.215079 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:39.215091 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:39.215124 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:39.217798 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:39.218398 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:39.218404 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:39.218406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:39.218544 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:39.218875 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:39.218878 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:39.218880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:39.219192 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:39.219418 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:39.219759 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 20:17:39.219956 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":344,"threads_started":1}
Aug 09 20:17:39 dist-test-slave-6k22 krb5kdc[9519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:39.237685 10115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:39.224662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35302 (local address 127.5.61.62:39551)
0809 20:17:39.224834 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:39.224839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:39.225097 (+ 258us) server_negotiation.cc:407] Connection header received
0809 20:17:39.225255 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:39.225259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:39.225320 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:39.225401 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:39.226591 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:39.227262 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:39.228448 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:39.228862 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:39.228953 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:39.232781 (+ 3828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:39.232787 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:39.232790 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:39.232820 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:39.235322 (+ 2502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:39.235968 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:39.235971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:39.235973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:39.236085 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:39.236480 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:39.236483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:39.236485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:39.236841 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:39.237029 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:39.237298 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 20:17:39.237457 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 20:17:39.238246 9570 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35302
I20260809 20:17:39.239334 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 9645
I20260809 20:17:39.251807 10121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:39.240353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38726 (local address 127.5.61.1:35105)
0809 20:17:39.240586 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:39.240592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:39.240783 (+ 191us) server_negotiation.cc:407] Connection header received
0809 20:17:39.240966 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:39.240971 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:39.241045 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:39.241141 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:39.242404 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:39.243108 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:39.244372 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:39.244804 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:39.244937 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:39.247081 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:39.247086 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:39.247092 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:39.247127 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:39.249343 (+ 2216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:39.249935 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:39.249942 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:39.249945 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:39.250068 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:39.250467 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:39.250472 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:39.250475 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:39.250723 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:39.250984 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:39.251270 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 20:17:39.251474 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 20:17:39.252440 9786 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:39.334271 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 9645
I20260809 20:17:39.355926 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 9801
Aug 09 20:17:39 dist-test-slave-6k22 krb5kdc[9519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.2@KRBTEST.COM
I20260809 20:17:39.371467 10128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:39.356864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49838 (local address 127.5.61.2:45671)
0809 20:17:39.357493 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:39.357501 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:39.357527 (+ 26us) server_negotiation.cc:407] Connection header received
0809 20:17:39.357589 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:39.357595 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:39.357832 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:39.357960 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:39.359154 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:39.360291 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:39.361452 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:39.361990 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:39.362086 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:39.366181 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:39.366191 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:39.366205 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:39.366245 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:39.369138 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:39.369732 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:39.369739 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:39.369741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:39.369850 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:39.370156 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:39.370159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:39.370161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:39.370480 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:39.370706 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:39.370940 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 20:17:39.371148 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":333,"threads_started":1}
I20260809 20:17:39.372812 9951 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:39.456586 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 9801
I20260809 20:17:39.479055 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 9962
Aug 09 20:17:39 dist-test-slave-6k22 krb5kdc[9519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.3@KRBTEST.COM
I20260809 20:17:39.495971 10130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:39.480172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41270 (local address 127.5.61.3:33279)
0809 20:17:39.480935 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:39.480942 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:39.480973 (+ 31us) server_negotiation.cc:407] Connection header received
0809 20:17:39.481033 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:39.481040 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:39.481262 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:39.481453 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:39.482811 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:39.483981 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:39.485304 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:39.485880 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:39.486023 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:39.490375 (+ 4352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:39.490388 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:39.490402 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:39.490436 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:39.493399 (+ 2963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:39.494074 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:39.494080 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:39.494082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:39.494221 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:39.494535 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:39.494538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:39.494540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:39.494871 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:39.495111 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:39.495328 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 20:17:39.495527 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":413,"threads_started":1}
I20260809 20:17:39.497259 10104 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:39.579483 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 9962
I20260809 20:17:39.600093 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 9535
I20260809 20:17:39.601271 9610 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:39.703486 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 9535
2026-08-09T20:17:39Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4011 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:17:39 dist-test-slave-6k22 krb5kdc[10136](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:17:39 dist-test-slave-6k22 krb5kdc[10136](info): set up 2 sockets
Aug 09 20:17:39 dist-test-slave-6k22 krb5kdc[10136](info): commencing operation
krb5kdc: starting...
W20260809 20:17:40.859716 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:17:40 dist-test-slave-6k22 krb5kdc[10136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:17:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:17:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:17:41.067839 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:38235
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:33285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:38235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:41.379772 10153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:41.380060 10153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:41.380234 10153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:41.387324 10153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:17:41.387427 10153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:41.387473 10153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:41.387552 10153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:17:41.387665 10153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:17:41.397370 10153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:33285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:38235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:38235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:41.398669 10153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:41.400023 10153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:41.411748 10159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:41.411711 10160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:41.411911 10162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:41.412575 10153 server_base.cc:1034] running on GCE node
I20260809 20:17:41.413362 10153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:41.414647 10153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:41.415838 10153 hybrid_clock.cc:648] HybridClock initialized: now 1786306661415782 us; error 54 us; skew 500 ppm
I20260809 20:17:41.416222 10153 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:41 dist-test-slave-6k22 krb5kdc[10136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306661, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:41.419394 10153 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:17:41.420542 10153 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:41.420630 10153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:41.420898 10153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:41.423746 10153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "1ca69f431d214349b1e15763756c8096"
format_stamp: "Formatted at 2026-08-09 20:17:41 on dist-test-slave-6k22"
server_key: "021389ecc97b9ce14dabef9a9a0f3218"
server_key_iv: "173d2828343e97fd8ee536b59529ab2f"
server_key_version: "encryptionkey@0"
I20260809 20:17:41.424522 10153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "1ca69f431d214349b1e15763756c8096"
format_stamp: "Formatted at 2026-08-09 20:17:41 on dist-test-slave-6k22"
server_key: "021389ecc97b9ce14dabef9a9a0f3218"
server_key_iv: "173d2828343e97fd8ee536b59529ab2f"
server_key_version: "encryptionkey@0"
I20260809 20:17:41.429394 10153 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 20:17:41.432792 10170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:41.434049 10153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:17:41.434230 10153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "1ca69f431d214349b1e15763756c8096"
format_stamp: "Formatted at 2026-08-09 20:17:41 on dist-test-slave-6k22"
server_key: "021389ecc97b9ce14dabef9a9a0f3218"
server_key_iv: "173d2828343e97fd8ee536b59529ab2f"
server_key_version: "encryptionkey@0"
I20260809 20:17:41.434451 10153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:41.446895 10153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:41.451081 10153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:41.451345 10153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:41.469161 10153 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:38235
I20260809 20:17:41.469188 10226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:38235 every 8 connection(s)
I20260809 20:17:41.470049 10153 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:17:41.473605 10227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:41.477202 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 10153
I20260809 20:17:41.477448 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:17:41.478067 5364 external_mini_cluster.cc:1428] Setting key 2839a3c6e351b6cb6781c5b0b0251832
I20260809 20:17:41.485517 10227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ca69f431d214349b1e15763756c8096: Bootstrap starting.
I20260809 20:17:41.489214 10227 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ca69f431d214349b1e15763756c8096: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:41.490425 10227 log.cc:824] T 00000000000000000000000000000000 P 1ca69f431d214349b1e15763756c8096: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:41.493225 10227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ca69f431d214349b1e15763756c8096: No bootstrap required, opened a new log
Aug 09 20:17:41 dist-test-slave-6k22 krb5kdc[10136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:41.502426 10230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:41.480072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56408 (local address 127.5.61.62:38235)
0809 20:17:41.480935 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:41.480952 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:41.481002 (+ 50us) server_negotiation.cc:407] Connection header received
0809 20:17:41.483872 (+ 2870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:41.483898 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:41.484347 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:41.484754 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:41.486166 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:41.488293 (+ 2127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:41.489714 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:41.490755 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:41.490958 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:41.495448 (+ 4490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:41.495458 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:41.495470 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:41.495517 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:41.498591 (+ 3074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:41.499368 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:41.499373 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:41.499379 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:41.499536 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:41.500003 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:41.500006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:41.500008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:41.500518 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:41.500711 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:41.501154 (+ 443us) server_negotiation.cc:299] Negotiation successful
0809 20:17:41.501475 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":363,"threads_started":1}
I20260809 20:17:41.502359 10227 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ca69f431d214349b1e15763756c8096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca69f431d214349b1e15763756c8096" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38235 } }
I20260809 20:17:41.502759 10227 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ca69f431d214349b1e15763756c8096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:41.502895 10227 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ca69f431d214349b1e15763756c8096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca69f431d214349b1e15763756c8096, State: Initialized, Role: FOLLOWER
I20260809 20:17:41.503515 10227 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ca69f431d214349b1e15763756c8096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca69f431d214349b1e15763756c8096" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38235 } }
I20260809 20:17:41.503808 10227 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ca69f431d214349b1e15763756c8096 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:17:41.503959 10227 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ca69f431d214349b1e15763756c8096 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:17:41.504170 10227 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ca69f431d214349b1e15763756c8096 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:41.506553 10227 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ca69f431d214349b1e15763756c8096 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca69f431d214349b1e15763756c8096" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38235 } }
I20260809 20:17:41.507023 10227 leader_election.cc:302] T 00000000000000000000000000000000 P 1ca69f431d214349b1e15763756c8096 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ca69f431d214349b1e15763756c8096; no voters:
I20260809 20:17:41.507650 10227 leader_election.cc:288] T 00000000000000000000000000000000 P 1ca69f431d214349b1e15763756c8096 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:17:41.507776 10232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ca69f431d214349b1e15763756c8096 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:41.508934 10232 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ca69f431d214349b1e15763756c8096 [term 1 LEADER]: Becoming Leader. State: Replica: 1ca69f431d214349b1e15763756c8096, State: Running, Role: LEADER
I20260809 20:17:41.509476 10232 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ca69f431d214349b1e15763756c8096 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca69f431d214349b1e15763756c8096" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38235 } }
I20260809 20:17:41.509793 10227 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ca69f431d214349b1e15763756c8096 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:17:41.512359 10235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ca69f431d214349b1e15763756c8096 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ca69f431d214349b1e15763756c8096. Latest consensus state: current_term: 1 leader_uuid: "1ca69f431d214349b1e15763756c8096" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca69f431d214349b1e15763756c8096" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38235 } } }
I20260809 20:17:41.512759 10235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ca69f431d214349b1e15763756c8096 [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:41.513393 10233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ca69f431d214349b1e15763756c8096 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ca69f431d214349b1e15763756c8096" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca69f431d214349b1e15763756c8096" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38235 } } }
I20260809 20:17:41.513765 10233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ca69f431d214349b1e15763756c8096 [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:41.514302 10240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:17:41.520215 10240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:17:41.529114 10240 catalog_manager.cc:1347] Generated new cluster ID: fa8705d71934400da9364952650d5255
I20260809 20:17:41.529223 10240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:17:41.558523 10240 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:17:41.560415 10240 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:17:41.571386 10240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ca69f431d214349b1e15763756c8096: Generated new TSK 0
I20260809 20:17:41.572402 10240 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.61.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
I20260809 20:17:41.630944 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:38235
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:33285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:41.961432 10259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:41.961752 10259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:41.961951 10259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:41.969314 10259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:41.969481 10259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:41.969686 10259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:17:41.980188 10259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:33285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:38235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:41.981541 10259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:41.982877 10259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:41.995781 10268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:41.995893 10264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:41.995936 10266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:41.996855 10259 server_base.cc:1034] running on GCE node
I20260809 20:17:41.997445 10259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:41.998347 10259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:41.999648 10259 hybrid_clock.cc:648] HybridClock initialized: now 1786306661999582 us; error 46 us; skew 500 ppm
I20260809 20:17:41.999971 10259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:41 dist-test-slave-6k22 krb5kdc[10136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306661, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:42.002985 10259 init.cc:377] Logged in from keytab as kudu/127.5.61.1@KRBTEST.COM (short username kudu)
I20260809 20:17:42.004204 10259 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:42.004339 10259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:42.004705 10259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:42.007753 10259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "917b18fde19b4c89a3cbd2ff8edc2691"
format_stamp: "Formatted at 2026-08-09 20:17:42 on dist-test-slave-6k22"
server_key: "2b7540445febe5c2cfc29d3b221ea372"
server_key_iv: "ee7537fff4614b1ef9d5306a7609e80b"
server_key_version: "encryptionkey@0"
I20260809 20:17:42.008601 10259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "917b18fde19b4c89a3cbd2ff8edc2691"
format_stamp: "Formatted at 2026-08-09 20:17:42 on dist-test-slave-6k22"
server_key: "2b7540445febe5c2cfc29d3b221ea372"
server_key_iv: "ee7537fff4614b1ef9d5306a7609e80b"
server_key_version: "encryptionkey@0"
I20260809 20:17:42.013652 10259 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 20:17:42.017133 10276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:42.018394 10259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 20:17:42.018576 10259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "917b18fde19b4c89a3cbd2ff8edc2691"
format_stamp: "Formatted at 2026-08-09 20:17:42 on dist-test-slave-6k22"
server_key: "2b7540445febe5c2cfc29d3b221ea372"
server_key_iv: "ee7537fff4614b1ef9d5306a7609e80b"
server_key_version: "encryptionkey@0"
I20260809 20:17:42.018801 10259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:42.042111 10259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:42.046068 10259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:42.046378 10259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:42.047386 10259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:42.048840 10259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:42.048980 10259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:42.049130 10259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:42.049230 10259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:42.083316 10259 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:44981
I20260809 20:17:42.083403 10411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:44981 every 8 connection(s)
I20260809 20:17:42.084169 10259 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:17:42.091267 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 10259
I20260809 20:17:42.091488 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:17:42.092123 5364 external_mini_cluster.cc:1428] Setting key 015f6a6e75c1cfe8e5e8b71108348958
Aug 09 20:17:42 dist-test-slave-6k22 krb5kdc[10136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306661, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:42.104387 10414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:42.086528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:47113 (local address 127.5.61.62:38235)
0809 20:17:42.086957 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:42.086961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:42.088072 (+ 1111us) server_negotiation.cc:407] Connection header received
0809 20:17:42.089789 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:42.089794 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:42.089876 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:42.090004 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:42.091366 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:42.092154 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:42.093821 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:42.094398 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:42.094535 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:42.098523 (+ 3988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:42.098528 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:42.098531 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:42.098563 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:42.101173 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:42.101933 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:42.101938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:42.101942 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:42.102083 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:42.102486 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:42.102492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:42.102496 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:42.102891 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:42.103039 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:42.103688 (+ 649us) server_negotiation.cc:299] Negotiation successful
0809 20:17:42.103939 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":206,"threads_started":1}
I20260809 20:17:42.105180 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:42.087094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38235 (local address 127.5.61.1:47113)
0809 20:17:42.087909 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:42.087957 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:42.089532 (+ 1575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:42.090223 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:42.090231 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:42.090712 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:42.091202 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:42.091219 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:42.092340 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:42.092346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:42.093653 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:42.093666 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:42.094596 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:42.094601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:42.094798 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:42.095496 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:42.095517 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:42.098287 (+ 2770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:42.101312 (+ 3025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:42.101324 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:42.101342 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:42.101807 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:42.102187 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:42.102208 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:42.102210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:42.102382 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:42.103008 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:42.103014 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:42.103302 (+ 288us) client_negotiation.cc:769] Sending connection context
0809 20:17:42.103531 (+ 229us) client_negotiation.cc:240] Negotiation successful
0809 20:17:42.103957 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":309,"threads_started":1}
I20260809 20:17:42.106756 10412 heartbeater.cc:347] Connected to a master server at 127.5.61.62:38235
I20260809 20:17:42.107061 10412 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:42.107750 10412 heartbeater.cc:511] Master 127.5.61.62:38235 requested a full tablet report, sending...
I20260809 20:17:42.109941 10190 ts_manager.cc:194] Registered new tserver with Master: 917b18fde19b4c89a3cbd2ff8edc2691 (127.5.61.1:44981)
I20260809 20:17:42.111526 10190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.1@KRBTEST.COM'} at 127.5.61.1:47113
WARNING: no policy specified for kudu/127.5.61.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
I20260809 20:17:42.142223 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:38235
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:33285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:42.474436 10418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:42.474740 10418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:42.474918 10418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:42.482040 10418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:42.482188 10418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:42.482359 10418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:17:42.491994 10418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:33285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:38235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:42.493255 10418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:42.494535 10418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:42.507685 10425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:42.507716 10423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:42.507925 10428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:42.508391 10418 server_base.cc:1034] running on GCE node
I20260809 20:17:42.508961 10418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:42.510017 10418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:42.511250 10418 hybrid_clock.cc:648] HybridClock initialized: now 1786306662511179 us; error 50 us; skew 500 ppm
I20260809 20:17:42.511610 10418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:42 dist-test-slave-6k22 krb5kdc[10136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306662, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:42.514572 10418 init.cc:377] Logged in from keytab as kudu/127.5.61.2@KRBTEST.COM (short username kudu)
I20260809 20:17:42.515820 10418 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:42.515954 10418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:42.516281 10418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:42.519219 10418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "68367c2f352a491786b2a0906313f4dc"
format_stamp: "Formatted at 2026-08-09 20:17:42 on dist-test-slave-6k22"
server_key: "1706d1782db7ea61f3c107d0b55c56d0"
server_key_iv: "9fb1a10e05cd44fb38a50a52f7e71222"
server_key_version: "encryptionkey@0"
I20260809 20:17:42.520118 10418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "68367c2f352a491786b2a0906313f4dc"
format_stamp: "Formatted at 2026-08-09 20:17:42 on dist-test-slave-6k22"
server_key: "1706d1782db7ea61f3c107d0b55c56d0"
server_key_iv: "9fb1a10e05cd44fb38a50a52f7e71222"
server_key_version: "encryptionkey@0"
I20260809 20:17:42.525194 10418 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 20:17:42.528463 10434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:42.529763 10418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:17:42.529902 10418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "68367c2f352a491786b2a0906313f4dc"
format_stamp: "Formatted at 2026-08-09 20:17:42 on dist-test-slave-6k22"
server_key: "1706d1782db7ea61f3c107d0b55c56d0"
server_key_iv: "9fb1a10e05cd44fb38a50a52f7e71222"
server_key_version: "encryptionkey@0"
I20260809 20:17:42.530076 10418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:42.542053 10418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:42.545778 10418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:42.545994 10418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:42.546838 10418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:42.548130 10418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:42.548251 10418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:42.548345 10418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:42.548401 10418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:42.581800 10418 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:36325
I20260809 20:17:42.581853 10568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:36325 every 8 connection(s)
I20260809 20:17:42.582652 10418 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:17:42.591405 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 10418
I20260809 20:17:42.591634 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:17:42.592186 5364 external_mini_cluster.cc:1428] Setting key 3d2cfb52079dc04bd9eb2dfa9f767cfa
Aug 09 20:17:42 dist-test-slave-6k22 krb5kdc[10136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306662, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:42.604663 10414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:42.585086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:53379 (local address 127.5.61.62:38235)
0809 20:17:42.585287 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:42.585293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:42.586582 (+ 1289us) server_negotiation.cc:407] Connection header received
0809 20:17:42.588464 (+ 1882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:42.588468 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:42.588535 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:42.588624 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:42.590090 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:42.590780 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:42.593100 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:42.593732 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:42.593856 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:42.597793 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:42.597798 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:42.597801 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:42.597833 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:42.601360 (+ 3527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:42.602324 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:42.602329 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:42.602333 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:42.602482 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:42.602909 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:42.602914 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:42.602918 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:42.603188 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:42.603344 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:42.604065 (+ 721us) server_negotiation.cc:299] Negotiation successful
0809 20:17:42.604303 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 20:17:42.605885 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:42.585712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38235 (local address 127.5.61.2:53379)
0809 20:17:42.586417 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:42.586464 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:42.588221 (+ 1757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:42.588839 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:42.588847 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:42.589343 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:42.589916 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:42.589934 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:42.590915 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:42.590920 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:42.592946 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:42.592958 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:42.594019 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:42.594028 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:42.594256 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:42.594913 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:42.594939 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:42.597584 (+ 2645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:42.601521 (+ 3937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:42.601531 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:42.601549 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:42.602185 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:42.602596 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:42.602601 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:42.602605 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:42.602787 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:42.603320 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:42.603327 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:42.603697 (+ 370us) client_negotiation.cc:769] Sending connection context
0809 20:17:42.603968 (+ 271us) client_negotiation.cc:240] Negotiation successful
0809 20:17:42.604479 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":298,"threads_started":1}
I20260809 20:17:42.607436 10569 heartbeater.cc:347] Connected to a master server at 127.5.61.62:38235
I20260809 20:17:42.607796 10569 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:42.608477 10569 heartbeater.cc:511] Master 127.5.61.62:38235 requested a full tablet report, sending...
I20260809 20:17:42.609885 10190 ts_manager.cc:194] Registered new tserver with Master: 68367c2f352a491786b2a0906313f4dc (127.5.61.2:36325)
I20260809 20:17:42.610693 10190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.2@KRBTEST.COM'} at 127.5.61.2:53379
WARNING: no policy specified for kudu/127.5.61.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
I20260809 20:17:42.642817 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:38235
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:33285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:42.985836 10574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:42.986094 10574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:42.986199 10574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:42.993191 10574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:42.993297 10574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:42.993405 10574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:17:43.002671 10574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:33285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:38235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:43.003922 10574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:43.005125 10574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:43.018280 10580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:43.018335 10581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:43.018265 10583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:43.021080 10574 server_base.cc:1034] running on GCE node
I20260809 20:17:43.021657 10574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:43.022650 10574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:43.023875 10574 hybrid_clock.cc:648] HybridClock initialized: now 1786306663023832 us; error 57 us; skew 500 ppm
I20260809 20:17:43.024183 10574 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:43 dist-test-slave-6k22 krb5kdc[10136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306663, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:43.027240 10574 init.cc:377] Logged in from keytab as kudu/127.5.61.3@KRBTEST.COM (short username kudu)
I20260809 20:17:43.028595 10574 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:43.028725 10574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:43.029063 10574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:43.032042 10574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "8600f26fa4434b0cb0f22c9837a7dcac"
format_stamp: "Formatted at 2026-08-09 20:17:43 on dist-test-slave-6k22"
server_key: "9c472ec6ba5767e7a04b721efd4bfc33"
server_key_iv: "c2795c34d8d5c63b0fd544f83024ae24"
server_key_version: "encryptionkey@0"
I20260809 20:17:43.032893 10574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "8600f26fa4434b0cb0f22c9837a7dcac"
format_stamp: "Formatted at 2026-08-09 20:17:43 on dist-test-slave-6k22"
server_key: "9c472ec6ba5767e7a04b721efd4bfc33"
server_key_iv: "c2795c34d8d5c63b0fd544f83024ae24"
server_key_version: "encryptionkey@0"
I20260809 20:17:43.038007 10574 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 20:17:43.041419 10589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:43.042708 10574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 20:17:43.042889 10574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "8600f26fa4434b0cb0f22c9837a7dcac"
format_stamp: "Formatted at 2026-08-09 20:17:43 on dist-test-slave-6k22"
server_key: "9c472ec6ba5767e7a04b721efd4bfc33"
server_key_iv: "c2795c34d8d5c63b0fd544f83024ae24"
server_key_version: "encryptionkey@0"
I20260809 20:17:43.043113 10574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:43.066916 10574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:43.070641 10574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:43.070945 10574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:43.072003 10574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:43.073295 10574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:43.073441 10574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:43.073587 10574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:43.073674 10574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:43.107996 10574 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:33231
I20260809 20:17:43.108083 10726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:33231 every 8 connection(s)
I20260809 20:17:43.108839 10574 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:17:43.112715 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 10574
I20260809 20:17:43.112985 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:17:43.113552 5364 external_mini_cluster.cc:1428] Setting key b66d04ec907d4dcd8a615834d761d619
I20260809 20:17:43.114284 10412 heartbeater.cc:503] Master 127.5.61.62:38235 was elected leader, sending a full tablet report...
Aug 09 20:17:43 dist-test-slave-6k22 krb5kdc[10136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306663, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:43.129307 10730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:43.111383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:33603 (local address 127.5.61.62:38235)
0809 20:17:43.111953 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:43.111958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:43.114201 (+ 2243us) server_negotiation.cc:407] Connection header received
0809 20:17:43.115489 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:43.115495 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:43.115613 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:43.115771 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:43.117403 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.118150 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.119703 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.120138 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.120231 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:43.123531 (+ 3300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:43.123535 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:43.123539 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:43.123621 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:43.126333 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:43.126967 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:43.126971 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:43.126973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:43.127126 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:43.127602 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:43.127609 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:43.127617 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:43.127891 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:43.128062 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:43.128711 (+ 649us) server_negotiation.cc:299] Negotiation successful
0809 20:17:43.128945 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":296,"threads_started":1}
I20260809 20:17:43.130223 10731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:43.111856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38235 (local address 127.5.61.3:33603)
0809 20:17:43.112704 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:43.112765 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:43.115169 (+ 2404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:43.116000 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:43.116011 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:43.116554 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:43.117249 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:43.117266 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.118275 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.118278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:43.119499 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:43.119511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.120329 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.120334 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:43.120501 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:43.121107 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:43.121128 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:43.123276 (+ 2148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:43.126473 (+ 3197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:43.126479 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:43.126488 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:43.126869 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:43.127233 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:43.127242 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:43.127244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:43.127478 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:43.128034 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:43.128041 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:43.128347 (+ 306us) client_negotiation.cc:769] Sending connection context
0809 20:17:43.128603 (+ 256us) client_negotiation.cc:240] Negotiation successful
0809 20:17:43.129082 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":300,"threads_started":1}
I20260809 20:17:43.131541 10727 heartbeater.cc:347] Connected to a master server at 127.5.61.62:38235
I20260809 20:17:43.131886 10727 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:43.132472 10727 heartbeater.cc:511] Master 127.5.61.62:38235 requested a full tablet report, sending...
I20260809 20:17:43.133931 10189 ts_manager.cc:194] Registered new tserver with Master: 8600f26fa4434b0cb0f22c9837a7dcac (127.5.61.3:33231)
I20260809 20:17:43.134943 10189 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.3@KRBTEST.COM'} at 127.5.61.3:33603
I20260809 20:17:43.140918 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 20:17:43.156967 10730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:43.145279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56420 (local address 127.5.61.62:38235)
0809 20:17:43.145458 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:43.145462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:43.146000 (+ 538us) server_negotiation.cc:407] Connection header received
0809 20:17:43.146144 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:43.146148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:43.146215 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:43.146303 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:43.147889 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.148560 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.149813 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.150207 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.150297 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:43.152529 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:43.152533 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:43.152536 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:43.152561 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:43.154803 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:43.155359 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:43.155363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:43.155365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:43.155474 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:43.155850 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:43.155853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:43.155854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:43.156121 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:43.156303 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:43.156593 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 20:17:43.156769 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 20:17:43.160905 10189 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56420:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:17:43.167519 10189 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:17:43.199294 10744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:43.179209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:33231 (local address 127.0.0.1:55888)
0809 20:17:43.189692 (+ 10483us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:43.189728 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:43.189894 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:43.191366 (+ 1472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:43.191378 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:43.191421 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:43.192014 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:43.192027 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.194588 (+ 2561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.194592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:43.196670 (+ 2078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:43.196682 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.198196 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.198202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:43.198405 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:43.198478 (+ 73us) client_negotiation.cc:769] Sending connection context
0809 20:17:43.198614 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 20:17:43.198845 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10269,"thread_start_us":10130,"threads_started":1}
I20260809 20:17:43.199708 10742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:43.179288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55888 (local address 127.5.61.3:33231)
0809 20:17:43.190560 (+ 11272us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:43.190572 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:43.190620 (+ 48us) server_negotiation.cc:407] Connection header received
0809 20:17:43.190743 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:43.190755 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:43.191065 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:43.192260 (+ 1195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:43.192629 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.194441 (+ 1812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.196893 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.198017 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.198160 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:43.198411 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:43.199116 (+ 705us) server_negotiation.cc:299] Negotiation successful
0809 20:17:43.199297 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11036,"thread_start_us":15375,"threads_started":1}
I20260809 20:17:43.203441 10747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:43.190438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44496 (local address 127.5.61.2:36325)
0809 20:17:43.196296 (+ 5858us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:43.196304 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:43.196328 (+ 24us) server_negotiation.cc:407] Connection header received
0809 20:17:43.196438 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:43.196449 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:43.196682 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:43.196889 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:43.197460 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.198886 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.201084 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.202426 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.202568 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:43.202760 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:43.202990 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 20:17:43.203142 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5716,"thread_start_us":5565,"threads_started":1}
I20260809 20:17:43.203528 10743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:43.190131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:36325 (local address 127.0.0.1:44496)
0809 20:17:43.194686 (+ 4555us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:43.194704 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:43.194818 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:43.196891 (+ 2073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:43.196895 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:43.196938 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:43.197296 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:43.197306 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.199029 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.199035 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:43.200895 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:43.200913 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.202629 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.202635 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:43.202794 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:43.202831 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 20:17:43.202947 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 20:17:43.203076 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4399,"mutex_wait_us":8,"thread_start_us":4035,"threads_started":1}
I20260809 20:17:43.204135 10745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:43.179842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43916 (local address 127.5.61.1:44981)
0809 20:17:43.189320 (+ 9478us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:43.189333 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:43.194895 (+ 5562us) server_negotiation.cc:407] Connection header received
0809 20:17:43.195028 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:43.195037 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:43.195310 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:43.195633 (+ 323us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:43.196985 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.198505 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.201131 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.202498 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.202621 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:43.202824 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:43.203667 (+ 843us) server_negotiation.cc:299] Negotiation successful
0809 20:17:43.203829 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9249,"thread_start_us":9072,"threads_started":1}
I20260809 20:17:43.203858 10648 tablet_service.cc:1467] Processing CreateTablet for tablet 98167e43fb744bca879f65f7414b9928 (DEFAULT_TABLE table=test-table [id=3053de65ec8e47758af509d009eb80ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:43.204625 10746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:43.179251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:44981 (local address 127.0.0.1:43916)
0809 20:17:43.193979 (+ 14728us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:43.193996 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:43.194104 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:43.196275 (+ 2171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:43.196279 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:43.196307 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:43.196777 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:43.196793 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.198642 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.198647 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:43.200977 (+ 2330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:43.201011 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.202984 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.202990 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:43.203157 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:43.203195 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 20:17:43.203275 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 20:17:43.203405 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14617,"mutex_wait_us":23,"thread_start_us":15162,"threads_started":1}
I20260809 20:17:43.205516 10648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98167e43fb744bca879f65f7414b9928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:43.206871 10493 tablet_service.cc:1467] Processing CreateTablet for tablet 98167e43fb744bca879f65f7414b9928 (DEFAULT_TABLE table=test-table [id=3053de65ec8e47758af509d009eb80ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:43.208125 10493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98167e43fb744bca879f65f7414b9928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:43.208125 10335 tablet_service.cc:1467] Processing CreateTablet for tablet 98167e43fb744bca879f65f7414b9928 (DEFAULT_TABLE table=test-table [id=3053de65ec8e47758af509d009eb80ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:43.209427 10335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98167e43fb744bca879f65f7414b9928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:43.220502 10748 tablet_bootstrap.cc:492] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc: Bootstrap starting.
I20260809 20:17:43.223894 10748 tablet_bootstrap.cc:654] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:43.224953 10748 log.cc:824] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:43.226353 10750 tablet_bootstrap.cc:492] T 98167e43fb744bca879f65f7414b9928 P 917b18fde19b4c89a3cbd2ff8edc2691: Bootstrap starting.
I20260809 20:17:43.228068 10749 tablet_bootstrap.cc:492] T 98167e43fb744bca879f65f7414b9928 P 8600f26fa4434b0cb0f22c9837a7dcac: Bootstrap starting.
I20260809 20:17:43.228181 10748 tablet_bootstrap.cc:492] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc: No bootstrap required, opened a new log
I20260809 20:17:43.228559 10748 ts_tablet_manager.cc:1397] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.004s
I20260809 20:17:43.231740 10749 tablet_bootstrap.cc:654] T 98167e43fb744bca879f65f7414b9928 P 8600f26fa4434b0cb0f22c9837a7dcac: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:43.231753 10750 tablet_bootstrap.cc:654] T 98167e43fb744bca879f65f7414b9928 P 917b18fde19b4c89a3cbd2ff8edc2691: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:43.233028 10750 log.cc:824] T 98167e43fb744bca879f65f7414b9928 P 917b18fde19b4c89a3cbd2ff8edc2691: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:43.233028 10749 log.cc:824] T 98167e43fb744bca879f65f7414b9928 P 8600f26fa4434b0cb0f22c9837a7dcac: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:43.236495 10749 tablet_bootstrap.cc:492] T 98167e43fb744bca879f65f7414b9928 P 8600f26fa4434b0cb0f22c9837a7dcac: No bootstrap required, opened a new log
I20260809 20:17:43.236780 10749 ts_tablet_manager.cc:1397] T 98167e43fb744bca879f65f7414b9928 P 8600f26fa4434b0cb0f22c9837a7dcac: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260809 20:17:43.237138 10750 tablet_bootstrap.cc:492] T 98167e43fb744bca879f65f7414b9928 P 917b18fde19b4c89a3cbd2ff8edc2691: No bootstrap required, opened a new log
I20260809 20:17:43.237418 10750 ts_tablet_manager.cc:1397] T 98167e43fb744bca879f65f7414b9928 P 917b18fde19b4c89a3cbd2ff8edc2691: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.005s
I20260809 20:17:43.237293 10748 raft_consensus.cc:348] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8600f26fa4434b0cb0f22c9837a7dcac" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33231 } } peers { permanent_uuid: "917b18fde19b4c89a3cbd2ff8edc2691" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44981 } } peers { permanent_uuid: "68367c2f352a491786b2a0906313f4dc" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36325 } }
I20260809 20:17:43.237732 10748 raft_consensus.cc:374] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:43.237838 10748 raft_consensus.cc:729] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68367c2f352a491786b2a0906313f4dc, State: Initialized, Role: FOLLOWER
I20260809 20:17:43.238556 10748 consensus_queue.cc:260] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8600f26fa4434b0cb0f22c9837a7dcac" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33231 } } peers { permanent_uuid: "917b18fde19b4c89a3cbd2ff8edc2691" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44981 } } peers { permanent_uuid: "68367c2f352a491786b2a0906313f4dc" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36325 } }
I20260809 20:17:43.239638 10569 heartbeater.cc:503] Master 127.5.61.62:38235 was elected leader, sending a full tablet report...
I20260809 20:17:43.240247 10748 ts_tablet_manager.cc:1428] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc: Time spent starting tablet: real 0.012s user 0.010s sys 0.001s
I20260809 20:17:43.247534 10749 raft_consensus.cc:348] T 98167e43fb744bca879f65f7414b9928 P 8600f26fa4434b0cb0f22c9837a7dcac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8600f26fa4434b0cb0f22c9837a7dcac" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33231 } } peers { permanent_uuid: "917b18fde19b4c89a3cbd2ff8edc2691" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44981 } } peers { permanent_uuid: "68367c2f352a491786b2a0906313f4dc" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36325 } }
I20260809 20:17:43.247998 10749 raft_consensus.cc:374] T 98167e43fb744bca879f65f7414b9928 P 8600f26fa4434b0cb0f22c9837a7dcac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:43.248123 10749 raft_consensus.cc:729] T 98167e43fb744bca879f65f7414b9928 P 8600f26fa4434b0cb0f22c9837a7dcac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8600f26fa4434b0cb0f22c9837a7dcac, State: Initialized, Role: FOLLOWER
I20260809 20:17:43.248185 10750 raft_consensus.cc:348] T 98167e43fb744bca879f65f7414b9928 P 917b18fde19b4c89a3cbd2ff8edc2691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8600f26fa4434b0cb0f22c9837a7dcac" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33231 } } peers { permanent_uuid: "917b18fde19b4c89a3cbd2ff8edc2691" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44981 } } peers { permanent_uuid: "68367c2f352a491786b2a0906313f4dc" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36325 } }
I20260809 20:17:43.248594 10750 raft_consensus.cc:374] T 98167e43fb744bca879f65f7414b9928 P 917b18fde19b4c89a3cbd2ff8edc2691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:43.248698 10750 raft_consensus.cc:729] T 98167e43fb744bca879f65f7414b9928 P 917b18fde19b4c89a3cbd2ff8edc2691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 917b18fde19b4c89a3cbd2ff8edc2691, State: Initialized, Role: FOLLOWER
I20260809 20:17:43.248735 10749 consensus_queue.cc:260] T 98167e43fb744bca879f65f7414b9928 P 8600f26fa4434b0cb0f22c9837a7dcac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8600f26fa4434b0cb0f22c9837a7dcac" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33231 } } peers { permanent_uuid: "917b18fde19b4c89a3cbd2ff8edc2691" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44981 } } peers { permanent_uuid: "68367c2f352a491786b2a0906313f4dc" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36325 } }
I20260809 20:17:43.249289 10750 consensus_queue.cc:260] T 98167e43fb744bca879f65f7414b9928 P 917b18fde19b4c89a3cbd2ff8edc2691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8600f26fa4434b0cb0f22c9837a7dcac" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33231 } } peers { permanent_uuid: "917b18fde19b4c89a3cbd2ff8edc2691" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44981 } } peers { permanent_uuid: "68367c2f352a491786b2a0906313f4dc" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36325 } }
I20260809 20:17:43.249755 10727 heartbeater.cc:503] Master 127.5.61.62:38235 was elected leader, sending a full tablet report...
I20260809 20:17:43.250443 10749 ts_tablet_manager.cc:1428] T 98167e43fb744bca879f65f7414b9928 P 8600f26fa4434b0cb0f22c9837a7dcac: Time spent starting tablet: real 0.014s user 0.007s sys 0.007s
I20260809 20:17:43.250909 10750 ts_tablet_manager.cc:1428] T 98167e43fb744bca879f65f7414b9928 P 917b18fde19b4c89a3cbd2ff8edc2691: Time spent starting tablet: real 0.013s user 0.007s sys 0.008s
W20260809 20:17:43.334606 10570 tablet.cc:2369] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 20:17:43.336390 10413 tablet.cc:2369] T 98167e43fb744bca879f65f7414b9928 P 917b18fde19b4c89a3cbd2ff8edc2691: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 20:17:43.360391 10728 tablet.cc:2369] T 98167e43fb744bca879f65f7414b9928 P 8600f26fa4434b0cb0f22c9837a7dcac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:17:43.427136 10754 raft_consensus.cc:482] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:17:43.427460 10754 raft_consensus.cc:504] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8600f26fa4434b0cb0f22c9837a7dcac" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33231 } } peers { permanent_uuid: "917b18fde19b4c89a3cbd2ff8edc2691" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44981 } } peers { permanent_uuid: "68367c2f352a491786b2a0906313f4dc" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36325 } }
I20260809 20:17:43.429333 10754 leader_election.cc:288] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8600f26fa4434b0cb0f22c9837a7dcac (127.5.61.3:33231), 917b18fde19b4c89a3cbd2ff8edc2691 (127.5.61.1:44981)
I20260809 20:17:43.436656 10745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:43.429900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:33345 (local address 127.5.61.1:44981)
0809 20:17:43.430266 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:43.430272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:43.430719 (+ 447us) server_negotiation.cc:407] Connection header received
0809 20:17:43.430900 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:43.430906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:43.430984 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:43.431086 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:43.431791 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.432638 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.434778 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.435622 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.435738 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:43.435812 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:43.436222 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 20:17:43.436373 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217}
I20260809 20:17:43.436704 10742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:43.429795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:42719 (local address 127.5.61.3:33231)
0809 20:17:43.430074 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:43.430084 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:43.430905 (+ 821us) server_negotiation.cc:407] Connection header received
0809 20:17:43.431113 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:43.431118 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:43.431200 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:43.431317 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:43.432054 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.432846 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.434883 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.435775 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.435905 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:43.436007 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:43.436323 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 20:17:43.436465 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260809 20:17:43.436748 10758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:43.429582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:44981 (local address 127.5.61.2:33345)
0809 20:17:43.430583 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:43.430607 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:43.430741 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:43.431110 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:43.431114 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:43.431140 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:43.431635 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:43.431655 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.432782 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.432790 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:43.434624 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:43.434639 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.435857 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.435866 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:43.436058 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:43.436085 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 20:17:43.436163 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 20:17:43.436274 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":854,"mutex_wait_us":70,"thread_start_us":873,"threads_started":1}
I20260809 20:17:43.436818 10757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:43.429636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:33231 (local address 127.5.61.2:42719)
0809 20:17:43.430764 (+ 1128us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:43.430787 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:43.430950 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:43.431341 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:43.431344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:43.431365 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:43.431853 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:43.431879 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.433015 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.433023 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:43.434683 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:43.434719 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.435990 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.435995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:43.436123 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:43.436176 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 20:17:43.436274 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 20:17:43.436388 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":980,"mutex_wait_us":44,"thread_start_us":510,"threads_started":1}
I20260809 20:17:43.437944 10676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98167e43fb744bca879f65f7414b9928" candidate_uuid: "68367c2f352a491786b2a0906313f4dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600f26fa4434b0cb0f22c9837a7dcac" is_pre_election: true
I20260809 20:17:43.437945 10357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98167e43fb744bca879f65f7414b9928" candidate_uuid: "68367c2f352a491786b2a0906313f4dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "917b18fde19b4c89a3cbd2ff8edc2691" is_pre_election: true
I20260809 20:17:43.438572 10357 raft_consensus.cc:2454] T 98167e43fb744bca879f65f7414b9928 P 917b18fde19b4c89a3cbd2ff8edc2691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68367c2f352a491786b2a0906313f4dc in term 0.
I20260809 20:17:43.438571 10676 raft_consensus.cc:2454] T 98167e43fb744bca879f65f7414b9928 P 8600f26fa4434b0cb0f22c9837a7dcac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68367c2f352a491786b2a0906313f4dc in term 0.
I20260809 20:17:43.439288 10436 leader_election.cc:302] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68367c2f352a491786b2a0906313f4dc, 917b18fde19b4c89a3cbd2ff8edc2691; no voters:
I20260809 20:17:43.439718 10754 raft_consensus.cc:2781] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:17:43.439884 10754 raft_consensus.cc:482] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:17:43.439994 10754 raft_consensus.cc:3037] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:43.442075 10754 raft_consensus.cc:504] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8600f26fa4434b0cb0f22c9837a7dcac" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33231 } } peers { permanent_uuid: "917b18fde19b4c89a3cbd2ff8edc2691" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44981 } } peers { permanent_uuid: "68367c2f352a491786b2a0906313f4dc" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36325 } }
I20260809 20:17:43.442636 10754 leader_election.cc:288] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc [CANDIDATE]: Term 1 election: Requested vote from peers 8600f26fa4434b0cb0f22c9837a7dcac (127.5.61.3:33231), 917b18fde19b4c89a3cbd2ff8edc2691 (127.5.61.1:44981)
I20260809 20:17:43.443066 10676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98167e43fb744bca879f65f7414b9928" candidate_uuid: "68367c2f352a491786b2a0906313f4dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600f26fa4434b0cb0f22c9837a7dcac"
I20260809 20:17:43.443197 10357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98167e43fb744bca879f65f7414b9928" candidate_uuid: "68367c2f352a491786b2a0906313f4dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "917b18fde19b4c89a3cbd2ff8edc2691"
I20260809 20:17:43.443305 10676 raft_consensus.cc:3037] T 98167e43fb744bca879f65f7414b9928 P 8600f26fa4434b0cb0f22c9837a7dcac [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:43.443395 10357 raft_consensus.cc:3037] T 98167e43fb744bca879f65f7414b9928 P 917b18fde19b4c89a3cbd2ff8edc2691 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:43.446621 10676 raft_consensus.cc:2454] T 98167e43fb744bca879f65f7414b9928 P 8600f26fa4434b0cb0f22c9837a7dcac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68367c2f352a491786b2a0906313f4dc in term 1.
I20260809 20:17:43.446638 10357 raft_consensus.cc:2454] T 98167e43fb744bca879f65f7414b9928 P 917b18fde19b4c89a3cbd2ff8edc2691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68367c2f352a491786b2a0906313f4dc in term 1.
I20260809 20:17:43.447055 10436 leader_election.cc:302] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68367c2f352a491786b2a0906313f4dc, 917b18fde19b4c89a3cbd2ff8edc2691; no voters:
I20260809 20:17:43.447337 10754 raft_consensus.cc:2781] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:43.447865 10754 raft_consensus.cc:686] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc [term 1 LEADER]: Becoming Leader. State: Replica: 68367c2f352a491786b2a0906313f4dc, State: Running, Role: LEADER
I20260809 20:17:43.448400 10754 consensus_queue.cc:237] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8600f26fa4434b0cb0f22c9837a7dcac" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33231 } } peers { permanent_uuid: "917b18fde19b4c89a3cbd2ff8edc2691" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44981 } } peers { permanent_uuid: "68367c2f352a491786b2a0906313f4dc" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36325 } }
I20260809 20:17:43.452598 10189 catalog_manager.cc:5495] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 68367c2f352a491786b2a0906313f4dc (127.5.61.2). New cstate: current_term: 1 leader_uuid: "68367c2f352a491786b2a0906313f4dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8600f26fa4434b0cb0f22c9837a7dcac" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "917b18fde19b4c89a3cbd2ff8edc2691" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68367c2f352a491786b2a0906313f4dc" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 36325 } health_report { overall_health: HEALTHY } } }
I20260809 20:17:43.484764 10747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:43.478846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44498 (local address 127.5.61.2:36325)
0809 20:17:43.479029 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:43.479034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:43.479074 (+ 40us) server_negotiation.cc:407] Connection header received
0809 20:17:43.479281 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:43.479286 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:43.479382 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:43.479530 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:17:43.479952 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.480992 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.482230 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.482890 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.483013 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:43.483457 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:17:43.484135 (+ 678us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:17:43.484281 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:43.484338 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 20:17:43.484416 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 20:17:43.493199 10676 raft_consensus.cc:1261] T 98167e43fb744bca879f65f7414b9928 P 8600f26fa4434b0cb0f22c9837a7dcac [term 1 FOLLOWER]: Refusing update from remote peer 68367c2f352a491786b2a0906313f4dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:17:43.493199 10357 raft_consensus.cc:1261] T 98167e43fb744bca879f65f7414b9928 P 917b18fde19b4c89a3cbd2ff8edc2691 [term 1 FOLLOWER]: Refusing update from remote peer 68367c2f352a491786b2a0906313f4dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:17:43.494151 10754 consensus_queue.cc:1035] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600f26fa4434b0cb0f22c9837a7dcac" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:17:43.494442 10759 consensus_queue.cc:1035] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "917b18fde19b4c89a3cbd2ff8edc2691" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:17:43.506222 10762 mvcc.cc:205] Tried to move back new op lower bound from 7316712093657657344 to 7316712093488443392. Current Snapshot: MvccSnapshot[applied={T|T < 7316712093657657344}]
I20260809 20:17:43.507426 10763 mvcc.cc:205] Tried to move back new op lower bound from 7316712093657657344 to 7316712093488443392. Current Snapshot: MvccSnapshot[applied={T|T < 7316712093657657344}]
I20260809 20:17:43.518458 10190 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:17:43.518707 10190 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:17:43.522177 10190 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1ca69f431d214349b1e15763756c8096: Sending DeleteTablet for 3 replicas of tablet 98167e43fb744bca879f65f7414b9928
I20260809 20:17:43.523425 10335 tablet_service.cc:1514] Processing DeleteTablet for tablet 98167e43fb744bca879f65f7414b9928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:17:43 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:43916
I20260809 20:17:43.523826 10648 tablet_service.cc:1514] Processing DeleteTablet for tablet 98167e43fb744bca879f65f7414b9928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:17:43 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:55888
I20260809 20:17:43.523449 10493 tablet_service.cc:1514] Processing DeleteTablet for tablet 98167e43fb744bca879f65f7414b9928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:17:43 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:44496
I20260809 20:17:43.524991 10775 tablet_replica.cc:330] T 98167e43fb744bca879f65f7414b9928 P 917b18fde19b4c89a3cbd2ff8edc2691: stopping tablet replica
I20260809 20:17:43.525511 10775 raft_consensus.cc:2229] T 98167e43fb744bca879f65f7414b9928 P 917b18fde19b4c89a3cbd2ff8edc2691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:17:43.525760 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 10259
I20260809 20:17:43.526031 10775 raft_consensus.cc:2258] T 98167e43fb744bca879f65f7414b9928 P 917b18fde19b4c89a3cbd2ff8edc2691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:43.526818 10777 tablet_replica.cc:330] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc: stopping tablet replica
I20260809 20:17:43.527197 10777 raft_consensus.cc:2229] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:17:43.527880 10777 raft_consensus.cc:2258] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:43.529248 10776 tablet_replica.cc:330] T 98167e43fb744bca879f65f7414b9928 P 8600f26fa4434b0cb0f22c9837a7dcac: stopping tablet replica
I20260809 20:17:43.529495 10775 ts_tablet_manager.cc:1905] T 98167e43fb744bca879f65f7414b9928 P 917b18fde19b4c89a3cbd2ff8edc2691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:43.529706 10776 raft_consensus.cc:2229] T 98167e43fb744bca879f65f7414b9928 P 8600f26fa4434b0cb0f22c9837a7dcac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:17:43.530208 10776 raft_consensus.cc:2258] T 98167e43fb744bca879f65f7414b9928 P 8600f26fa4434b0cb0f22c9837a7dcac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:43.532327 10777 ts_tablet_manager.cc:1905] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:43.532406 10776 ts_tablet_manager.cc:1905] T 98167e43fb744bca879f65f7414b9928 P 8600f26fa4434b0cb0f22c9837a7dcac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:43.537218 10775 ts_tablet_manager.cc:1918] T 98167e43fb744bca879f65f7414b9928 P 917b18fde19b4c89a3cbd2ff8edc2691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:17:43.537477 10775 log.cc:1196] T 98167e43fb744bca879f65f7414b9928 P 917b18fde19b4c89a3cbd2ff8edc2691: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/98167e43fb744bca879f65f7414b9928
I20260809 20:17:43.538208 10775 ts_tablet_manager.cc:1939] T 98167e43fb744bca879f65f7414b9928 P 917b18fde19b4c89a3cbd2ff8edc2691: Deleting consensus metadata
I20260809 20:17:43.539803 10776 ts_tablet_manager.cc:1918] T 98167e43fb744bca879f65f7414b9928 P 8600f26fa4434b0cb0f22c9837a7dcac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:17:43.540011 10776 log.cc:1196] T 98167e43fb744bca879f65f7414b9928 P 8600f26fa4434b0cb0f22c9837a7dcac: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/98167e43fb744bca879f65f7414b9928
I20260809 20:17:43.540445 10172 catalog_manager.cc:4841] TS 917b18fde19b4c89a3cbd2ff8edc2691 (127.5.61.1:44981): tablet 98167e43fb744bca879f65f7414b9928 (table test-table [id=3053de65ec8e47758af509d009eb80ee]) successfully deleted
I20260809 20:17:43.540582 10776 ts_tablet_manager.cc:1939] T 98167e43fb744bca879f65f7414b9928 P 8600f26fa4434b0cb0f22c9837a7dcac: Deleting consensus metadata
I20260809 20:17:43.542611 10171 catalog_manager.cc:4841] TS 8600f26fa4434b0cb0f22c9837a7dcac (127.5.61.3:33231): tablet 98167e43fb744bca879f65f7414b9928 (table test-table [id=3053de65ec8e47758af509d009eb80ee]) successfully deleted
Aug 09 20:17:43 dist-test-slave-6k22 krb5kdc[10136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.1@KRBTEST.COM
I20260809 20:17:43.545354 10777 ts_tablet_manager.cc:1918] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:17:43.545534 10777 log.cc:1196] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/98167e43fb744bca879f65f7414b9928
I20260809 20:17:43.546178 10777 ts_tablet_manager.cc:1939] T 98167e43fb744bca879f65f7414b9928 P 68367c2f352a491786b2a0906313f4dc: Deleting consensus metadata
I20260809 20:17:43.548067 10171 catalog_manager.cc:4841] TS 68367c2f352a491786b2a0906313f4dc (127.5.61.2:36325): tablet 98167e43fb744bca879f65f7414b9928 (table test-table [id=3053de65ec8e47758af509d009eb80ee]) successfully deleted
I20260809 20:17:43.553368 10745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:43.528326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43922 (local address 127.5.61.1:44981)
0809 20:17:43.528507 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:43.528513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:43.528532 (+ 19us) server_negotiation.cc:407] Connection header received
0809 20:17:43.528590 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:43.528595 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:43.528679 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:43.528783 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:43.532543 (+ 3760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.533653 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.537156 (+ 3503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.537875 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.537990 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:43.545987 (+ 7997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:43.546005 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:43.546025 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:43.546080 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:43.550305 (+ 4225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:43.551045 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:43.551055 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:43.551059 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:43.551243 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:43.551685 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:43.551691 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:43.551694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:43.552182 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:43.552489 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:43.552744 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 20:17:43.552977 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 20:17:43.554064 10407 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:43.661371 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 10259
I20260809 20:17:43.682302 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 10418
Aug 09 20:17:43 dist-test-slave-6k22 krb5kdc[10136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.2@KRBTEST.COM
I20260809 20:17:43.700109 10747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:43.683154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44514 (local address 127.5.61.2:36325)
0809 20:17:43.683323 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:43.683327 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:43.683347 (+ 20us) server_negotiation.cc:407] Connection header received
0809 20:17:43.683550 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:43.683594 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:43.683683 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:43.683772 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:43.685376 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.686306 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.688144 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.688645 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.688756 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:43.693793 (+ 5037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:43.693805 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:43.693820 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:43.693869 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:43.696983 (+ 3114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:43.697693 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:43.697704 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:43.697706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:43.697893 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:43.698281 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:43.698287 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:43.698290 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:43.698679 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:43.699066 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:43.699316 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 20:17:43.699752 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 20:17:43.700788 10564 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:43.819751 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 10418
I20260809 20:17:43.843868 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 10574
Aug 09 20:17:43 dist-test-slave-6k22 krb5kdc[10136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.3@KRBTEST.COM
I20260809 20:17:43.858287 10742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:43.844634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55892 (local address 127.5.61.3:33231)
0809 20:17:43.844828 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:43.844833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:43.844861 (+ 28us) server_negotiation.cc:407] Connection header received
0809 20:17:43.845002 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:43.845006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:43.845080 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:43.845163 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:43.846533 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.847289 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.848537 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:43.848953 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:43.849044 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:43.853197 (+ 4153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:43.853210 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:43.853226 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:43.853264 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:43.855859 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:43.856449 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:43.856456 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:43.856458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:43.856597 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:43.856981 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:43.856984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:43.856986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:43.857318 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:43.857534 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:43.857765 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 20:17:43.857930 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 20:17:43.858908 10721 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:43.946079 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 10574
I20260809 20:17:43.967629 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 10153
I20260809 20:17:43.968789 10222 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:44.077409 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 10153
2026-08-09T20:17:44Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4374 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T20:17:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:17:44Z Disabled control of system clock
I20260809 20:17:44.150306 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:41557
--webserver_interface=127.5.61.62
--webserver_port=0
--builtin_ntp_servers=127.5.61.20:40371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:41557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 20:17:44.472074 10786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:44.472373 10786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:44.472455 10786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:44.479792 10786 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:17:44.479898 10786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:44.479935 10786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:44.479990 10786 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:17:44.480033 10786 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:17:44.489149 10786 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:40371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:41557
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:41557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:44.490368 10786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:44.492141 10786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:44.503873 10799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:44.503904 10796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:44.504467 10793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:44.504822 10786 server_base.cc:1034] running on GCE node
I20260809 20:17:44.505582 10786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:44.506848 10786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:44.508045 10786 hybrid_clock.cc:648] HybridClock initialized: now 1786306664507982 us; error 49 us; skew 500 ppm
I20260809 20:17:44.508392 10786 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:17:44.519796 10786 webserver.cc:460] Webserver started at http://127.5.61.62:42035/ using document root <none> and password file <none>
I20260809 20:17:44.520494 10786 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:44.520613 10786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:44.520964 10786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:44.523921 10786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "6850f2c04c0041c4b3c7ccbee16efe02"
format_stamp: "Formatted at 2026-08-09 20:17:44 on dist-test-slave-6k22"
server_key: "a4712d4e6406b3284a2347d329f28cc9"
server_key_iv: "567c118828dd8cfe8cc6464028731b6e"
server_key_version: "encryptionkey@0"
I20260809 20:17:44.524699 10786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "6850f2c04c0041c4b3c7ccbee16efe02"
format_stamp: "Formatted at 2026-08-09 20:17:44 on dist-test-slave-6k22"
server_key: "a4712d4e6406b3284a2347d329f28cc9"
server_key_iv: "567c118828dd8cfe8cc6464028731b6e"
server_key_version: "encryptionkey@0"
I20260809 20:17:44.529681 10786 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 20:17:44.532853 10805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:44.534130 10786 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 20:17:44.534307 10786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "6850f2c04c0041c4b3c7ccbee16efe02"
format_stamp: "Formatted at 2026-08-09 20:17:44 on dist-test-slave-6k22"
server_key: "a4712d4e6406b3284a2347d329f28cc9"
server_key_iv: "567c118828dd8cfe8cc6464028731b6e"
server_key_version: "encryptionkey@0"
I20260809 20:17:44.534513 10786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:44.540102 10786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:44.540866 10786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:44.541172 10786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:44.558729 10786 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:41557
I20260809 20:17:44.558768 10863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:41557 every 8 connection(s)
I20260809 20:17:44.560024 10786 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:17:44.563491 10864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:44.567832 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 10786
I20260809 20:17:44.568027 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:17:44.568845 5364 external_mini_cluster.cc:1428] Setting key 8e5b07644e2c990260096df903d8a6e3
I20260809 20:17:44.575168 10864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6850f2c04c0041c4b3c7ccbee16efe02: Bootstrap starting.
I20260809 20:17:44.577196 10867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:44.570893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47548 (local address 127.5.61.62:41557)
0809 20:17:44.571980 (+ 1087us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:44.571990 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:44.572031 (+ 41us) server_negotiation.cc:407] Connection header received
0809 20:17:44.573714 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:44.573733 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:44.574090 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:44.574389 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:44.575618 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:44.575625 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:44.575637 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:17:44.575652 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:44.575714 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:17:44.575735 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:44.575905 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:44.576027 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 20:17:44.576258 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":401,"threads_started":1}
I20260809 20:17:44.578651 10864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6850f2c04c0041c4b3c7ccbee16efe02: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:44.579993 10864 log.cc:824] T 00000000000000000000000000000000 P 6850f2c04c0041c4b3c7ccbee16efe02: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:44.582590 10864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6850f2c04c0041c4b3c7ccbee16efe02: No bootstrap required, opened a new log
I20260809 20:17:44.589570 10864 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6850f2c04c0041c4b3c7ccbee16efe02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6850f2c04c0041c4b3c7ccbee16efe02" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41557 } }
I20260809 20:17:44.589926 10864 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6850f2c04c0041c4b3c7ccbee16efe02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:44.590041 10864 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6850f2c04c0041c4b3c7ccbee16efe02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6850f2c04c0041c4b3c7ccbee16efe02, State: Initialized, Role: FOLLOWER
I20260809 20:17:44.590552 10864 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6850f2c04c0041c4b3c7ccbee16efe02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6850f2c04c0041c4b3c7ccbee16efe02" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41557 } }
I20260809 20:17:44.590802 10864 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6850f2c04c0041c4b3c7ccbee16efe02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:17:44.590939 10864 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6850f2c04c0041c4b3c7ccbee16efe02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:17:44.591109 10864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6850f2c04c0041c4b3c7ccbee16efe02 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:44.593196 10864 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6850f2c04c0041c4b3c7ccbee16efe02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6850f2c04c0041c4b3c7ccbee16efe02" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41557 } }
I20260809 20:17:44.593688 10864 leader_election.cc:302] T 00000000000000000000000000000000 P 6850f2c04c0041c4b3c7ccbee16efe02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6850f2c04c0041c4b3c7ccbee16efe02; no voters:
I20260809 20:17:44.594309 10864 leader_election.cc:288] T 00000000000000000000000000000000 P 6850f2c04c0041c4b3c7ccbee16efe02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:17:44.594449 10869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6850f2c04c0041c4b3c7ccbee16efe02 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:44.595538 10869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6850f2c04c0041c4b3c7ccbee16efe02 [term 1 LEADER]: Becoming Leader. State: Replica: 6850f2c04c0041c4b3c7ccbee16efe02, State: Running, Role: LEADER
I20260809 20:17:44.596135 10869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6850f2c04c0041c4b3c7ccbee16efe02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6850f2c04c0041c4b3c7ccbee16efe02" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41557 } }
I20260809 20:17:44.596433 10864 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6850f2c04c0041c4b3c7ccbee16efe02 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:17:44.599045 10872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6850f2c04c0041c4b3c7ccbee16efe02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6850f2c04c0041c4b3c7ccbee16efe02. Latest consensus state: current_term: 1 leader_uuid: "6850f2c04c0041c4b3c7ccbee16efe02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6850f2c04c0041c4b3c7ccbee16efe02" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41557 } } }
I20260809 20:17:44.599545 10872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6850f2c04c0041c4b3c7ccbee16efe02 [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:44.599040 10870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6850f2c04c0041c4b3c7ccbee16efe02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6850f2c04c0041c4b3c7ccbee16efe02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6850f2c04c0041c4b3c7ccbee16efe02" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41557 } } }
I20260809 20:17:44.599740 10870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6850f2c04c0041c4b3c7ccbee16efe02 [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:44.601838 10878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:17:44.607420 10878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:17:44.616515 10878 catalog_manager.cc:1347] Generated new cluster ID: fae5d1d488e74ed1821b698dd9f91971
I20260809 20:17:44.616636 10878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:17:44.633081 10878 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:17:44.633576 10878 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:17:44.640743 10878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6850f2c04c0041c4b3c7ccbee16efe02: Generated new TSK 0
I20260809 20:17:44.641657 10878 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:17:44.647318 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:41557
--builtin_ntp_servers=127.5.61.20:40371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 20:17:44.978215 10891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:44.978531 10891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:44.978718 10891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:44.986196 10891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:44.986308 10891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:44.986509 10891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:17:44.995499 10891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:40371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:41557
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:44.996997 10891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:44.998970 10891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:45.013381 10900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:45.013314 10897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:45.013406 10896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:45.014230 10891 server_base.cc:1034] running on GCE node
I20260809 20:17:45.014814 10891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:45.015856 10891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:45.017050 10891 hybrid_clock.cc:648] HybridClock initialized: now 1786306665016987 us; error 54 us; skew 500 ppm
I20260809 20:17:45.017373 10891 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:17:45.019834 10891 webserver.cc:460] Webserver started at http://127.5.61.1:38815/ using document root <none> and password file <none>
I20260809 20:17:45.020574 10891 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:45.020685 10891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:45.021050 10891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:45.023993 10891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "7a241709573d455bb7b88b15600de9d7"
format_stamp: "Formatted at 2026-08-09 20:17:45 on dist-test-slave-6k22"
server_key: "ed88c58dd9552a5dbe046efb2267244a"
server_key_iv: "176430fa71b4f3f4e40fe8078e9be3b8"
server_key_version: "encryptionkey@0"
I20260809 20:17:45.024762 10891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "7a241709573d455bb7b88b15600de9d7"
format_stamp: "Formatted at 2026-08-09 20:17:45 on dist-test-slave-6k22"
server_key: "ed88c58dd9552a5dbe046efb2267244a"
server_key_iv: "176430fa71b4f3f4e40fe8078e9be3b8"
server_key_version: "encryptionkey@0"
I20260809 20:17:45.030014 10891 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 20:17:45.033584 10906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:45.035094 10891 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 20:17:45.035269 10891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "7a241709573d455bb7b88b15600de9d7"
format_stamp: "Formatted at 2026-08-09 20:17:45 on dist-test-slave-6k22"
server_key: "ed88c58dd9552a5dbe046efb2267244a"
server_key_iv: "176430fa71b4f3f4e40fe8078e9be3b8"
server_key_version: "encryptionkey@0"
I20260809 20:17:45.035519 10891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:45.041316 10891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:45.042099 10891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:45.042379 10891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:45.043452 10891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:45.044849 10891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:45.044940 10891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:45.045022 10891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:45.045077 10891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:45.077170 10891 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:37927
I20260809 20:17:45.077216 11036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:37927 every 8 connection(s)
I20260809 20:17:45.078513 10891 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:17:45.079291 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 10891
I20260809 20:17:45.079519 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:17:45.080293 5364 external_mini_cluster.cc:1428] Setting key c7a2efa7f37f0077942e44d1084d0e60
I20260809 20:17:45.083272 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:41557
--builtin_ntp_servers=127.5.61.20:40371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 20:17:45.087307 11039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:45.081193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:44843 (local address 127.5.61.62:41557)
0809 20:17:45.082060 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:45.082088 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:45.083066 (+ 978us) server_negotiation.cc:407] Connection header received
0809 20:17:45.084873 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:45.084881 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:45.084951 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:45.085096 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:45.086446 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:45.086454 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:45.086461 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:17:45.086484 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:45.086541 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:17:45.086561 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:45.086687 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:45.086862 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 20:17:45.086931 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":760,"thread_start_us":658,"threads_started":1}
I20260809 20:17:45.088063 11040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:45.081827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41557 (local address 127.5.61.1:44843)
0809 20:17:45.082902 (+ 1075us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:45.082946 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:45.084604 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:45.085311 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:45.085319 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:45.085425 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:45.086133 (+ 708us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 20:17:45.086135 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:45.086172 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 20:17:45.086268 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:45.086659 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:45.086666 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:45.086671 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 20:17:45.086880 (+ 209us) client_negotiation.cc:240] Negotiation successful
0809 20:17:45.087066 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":360,"threads_started":1}
I20260809 20:17:45.089645 11037 heartbeater.cc:347] Connected to a master server at 127.5.61.62:41557
I20260809 20:17:45.089948 11037 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:45.090512 11037 heartbeater.cc:511] Master 127.5.61.62:41557 requested a full tablet report, sending...
I20260809 20:17:45.092399 10827 ts_manager.cc:194] Registered new tserver with Master: 7a241709573d455bb7b88b15600de9d7 (127.5.61.1:37927)
W20260809 20:17:45.414911 11041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:45.415186 11041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:45.415277 11041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:45.422605 11041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:45.422713 11041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:45.422827 11041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:17:45.431934 11041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:40371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:41557
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:45.433125 11041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:45.434921 11041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:45.448244 11050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:45.448309 11052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:45.448268 11049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:45.449137 11041 server_base.cc:1034] running on GCE node
I20260809 20:17:45.449649 11041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:45.450462 11041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:45.451648 11041 hybrid_clock.cc:648] HybridClock initialized: now 1786306665451583 us; error 57 us; skew 500 ppm
I20260809 20:17:45.451956 11041 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:17:45.454180 11041 webserver.cc:460] Webserver started at http://127.5.61.2:41941/ using document root <none> and password file <none>
I20260809 20:17:45.454846 11041 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:45.454937 11041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:45.455191 11041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:45.457980 11041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "390d7f48e3a241878524e34d6bfc0c14"
format_stamp: "Formatted at 2026-08-09 20:17:45 on dist-test-slave-6k22"
server_key: "86a762e62c65d2e5b02e5f35819d2d92"
server_key_iv: "d05520acd4c03f611cf5a7d66b58bcad"
server_key_version: "encryptionkey@0"
I20260809 20:17:45.458717 11041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "390d7f48e3a241878524e34d6bfc0c14"
format_stamp: "Formatted at 2026-08-09 20:17:45 on dist-test-slave-6k22"
server_key: "86a762e62c65d2e5b02e5f35819d2d92"
server_key_iv: "d05520acd4c03f611cf5a7d66b58bcad"
server_key_version: "encryptionkey@0"
I20260809 20:17:45.463603 11041 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 20:17:45.466979 11059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:45.468286 11041 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 20:17:45.468468 11041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "390d7f48e3a241878524e34d6bfc0c14"
format_stamp: "Formatted at 2026-08-09 20:17:45 on dist-test-slave-6k22"
server_key: "86a762e62c65d2e5b02e5f35819d2d92"
server_key_iv: "d05520acd4c03f611cf5a7d66b58bcad"
server_key_version: "encryptionkey@0"
I20260809 20:17:45.468690 11041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:45.474310 11041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:45.475102 11041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:45.475365 11041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:45.476397 11041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:45.477771 11041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:45.477864 11041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:45.478015 11041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:45.478107 11041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:45.512306 11041 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:38247
I20260809 20:17:45.512399 11191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:38247 every 8 connection(s)
I20260809 20:17:45.513631 11041 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:17:45.515625 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 11041
I20260809 20:17:45.515820 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:17:45.516366 5364 external_mini_cluster.cc:1428] Setting key ac8d48cc064ff8cf9a04751fabb707b8
I20260809 20:17:45.519105 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:41557
--builtin_ntp_servers=127.5.61.20:40371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 20:17:45.522621 11039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:45.516511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:54477 (local address 127.5.61.62:41557)
0809 20:17:45.516739 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:45.516745 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:45.518495 (+ 1750us) server_negotiation.cc:407] Connection header received
0809 20:17:45.520329 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:45.520334 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:45.520417 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:45.520538 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:45.521860 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:45.521866 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:45.521870 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:17:45.521889 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:45.521931 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:17:45.521943 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:45.522065 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:45.522258 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 20:17:45.522327 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260809 20:17:45.523622 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:45.517189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41557 (local address 127.5.61.2:54477)
0809 20:17:45.518301 (+ 1112us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:45.518365 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:45.520060 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:45.520762 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:45.520770 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:45.520864 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:45.521564 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 20:17:45.521566 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:45.521603 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 20:17:45.521685 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:45.522040 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:45.522051 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:45.522059 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 20:17:45.522338 (+ 279us) client_negotiation.cc:240] Negotiation successful
0809 20:17:45.522593 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":392,"threads_started":1}
I20260809 20:17:45.526576 11192 heartbeater.cc:347] Connected to a master server at 127.5.61.62:41557
I20260809 20:17:45.526880 11192 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:45.527426 11192 heartbeater.cc:511] Master 127.5.61.62:41557 requested a full tablet report, sending...
I20260809 20:17:45.528836 10827 ts_manager.cc:194] Registered new tserver with Master: 390d7f48e3a241878524e34d6bfc0c14 (127.5.61.2:38247)
W20260809 20:17:45.849251 11195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:45.849589 11195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:45.849779 11195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:45.857057 11195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:45.857198 11195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:45.857398 11195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:17:45.866895 11195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:40371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:41557
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:45.868206 11195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:45.870074 11195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:45.883451 11201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:45.883457 11200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:45.883432 11203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:45.884234 11195 server_base.cc:1034] running on GCE node
I20260809 20:17:45.884809 11195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:45.885660 11195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:45.886874 11195 hybrid_clock.cc:648] HybridClock initialized: now 1786306665886822 us; error 43 us; skew 500 ppm
I20260809 20:17:45.887158 11195 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:17:45.889478 11195 webserver.cc:460] Webserver started at http://127.5.61.3:45743/ using document root <none> and password file <none>
I20260809 20:17:45.890193 11195 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:45.890290 11195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:45.890544 11195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:45.893348 11195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "d7d79746cb96421eb606b1d610db55d7"
format_stamp: "Formatted at 2026-08-09 20:17:45 on dist-test-slave-6k22"
server_key: "8e8129c951470af802b00645285c8b15"
server_key_iv: "c6f869f6c23cb70540d16e1759ad2b4f"
server_key_version: "encryptionkey@0"
I20260809 20:17:45.894059 11195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "d7d79746cb96421eb606b1d610db55d7"
format_stamp: "Formatted at 2026-08-09 20:17:45 on dist-test-slave-6k22"
server_key: "8e8129c951470af802b00645285c8b15"
server_key_iv: "c6f869f6c23cb70540d16e1759ad2b4f"
server_key_version: "encryptionkey@0"
I20260809 20:17:45.899035 11195 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 20:17:45.902462 11211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:45.903821 11195 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:17:45.903947 11195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "d7d79746cb96421eb606b1d610db55d7"
format_stamp: "Formatted at 2026-08-09 20:17:45 on dist-test-slave-6k22"
server_key: "8e8129c951470af802b00645285c8b15"
server_key_iv: "c6f869f6c23cb70540d16e1759ad2b4f"
server_key_version: "encryptionkey@0"
I20260809 20:17:45.904129 11195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:45.909452 11195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:45.910146 11195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:45.910375 11195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:45.911211 11195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:45.912725 11195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:45.912823 11195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:45.912900 11195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:45.912952 11195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:45.946800 11195 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:40447
I20260809 20:17:45.946854 11341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:40447 every 8 connection(s)
I20260809 20:17:45.948199 11195 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:17:45.951036 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 11195
I20260809 20:17:45.951217 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:17:45.951745 5364 external_mini_cluster.cc:1428] Setting key a4ab03e37b6d20d2289a2c6f0276a13f
I20260809 20:17:45.957391 11039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:45.950927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:45699 (local address 127.5.61.62:41557)
0809 20:17:45.951177 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:45.951184 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:45.952929 (+ 1745us) server_negotiation.cc:407] Connection header received
0809 20:17:45.955216 (+ 2287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:45.955219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:45.955270 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:45.955353 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:45.956770 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:45.956774 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:45.956777 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:17:45.956793 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:45.956825 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:17:45.956837 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:45.956943 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:45.957113 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 20:17:45.957161 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 20:17:45.958333 11344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:45.951530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41557 (local address 127.5.61.3:45699)
0809 20:17:45.952717 (+ 1187us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:45.952775 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:45.954900 (+ 2125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:45.955675 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:45.955684 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:45.955786 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:45.956473 (+ 687us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 20:17:45.956476 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:45.956514 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 20:17:45.956596 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:45.956934 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:45.956942 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:45.956948 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 20:17:45.957162 (+ 214us) client_negotiation.cc:240] Negotiation successful
0809 20:17:45.957376 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":358,"threads_started":1}
I20260809 20:17:45.959740 11342 heartbeater.cc:347] Connected to a master server at 127.5.61.62:41557
I20260809 20:17:45.960011 11342 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:45.960590 11342 heartbeater.cc:511] Master 127.5.61.62:41557 requested a full tablet report, sending...
I20260809 20:17:45.961987 10827 ts_manager.cc:194] Registered new tserver with Master: d7d79746cb96421eb606b1d610db55d7 (127.5.61.3:40447)
I20260809 20:17:45.967278 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 20:17:45.974537 11039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:45.972110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47564 (local address 127.5.61.62:41557)
0809 20:17:45.972316 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:45.972324 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:45.972811 (+ 487us) server_negotiation.cc:407] Connection header received
0809 20:17:45.972942 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:45.972945 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:45.972997 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:45.973092 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:45.973971 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:45.973975 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:45.973979 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:17:45.973993 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:45.974026 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:17:45.974038 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:45.974132 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:45.974184 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 20:17:45.974227 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 20:17:45.978441 10827 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:17:45.985965 10827 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:17:46.006157 11355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:45.998554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:40447 (local address 127.0.0.1:48624)
0809 20:17:46.000307 (+ 1753us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:46.000352 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:46.000484 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:46.003030 (+ 2546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:46.003042 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:46.003159 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:46.004209 (+ 1050us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 20:17:46.004215 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:46.004254 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 20:17:46.004410 (+ 156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:46.005080 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:46.005095 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:46.005107 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 20:17:46.005351 (+ 244us) client_negotiation.cc:240] Negotiation successful
0809 20:17:46.005481 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1529,"thread_start_us":1380,"threads_started":1}
I20260809 20:17:46.007130 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:45.998789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48624 (local address 127.5.61.3:40447)
0809 20:17:45.999737 (+ 948us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:45.999759 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:46.000637 (+ 878us) server_negotiation.cc:407] Connection header received
0809 20:17:46.000717 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:46.000725 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:46.001381 (+ 656us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:46.001578 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:46.004647 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:46.004660 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:46.004680 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:17:46.004707 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:46.004791 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:17:46.004818 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:46.005012 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:46.006459 (+ 1447us) server_negotiation.cc:299] Negotiation successful
0809 20:17:46.006597 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":440,"threads_started":1}
I20260809 20:17:46.012706 11357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:46.000751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:38247 (local address 127.0.0.1:52768)
0809 20:17:46.008487 (+ 7736us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:46.008515 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:46.008611 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:46.009828 (+ 1217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:46.009833 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:46.009854 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:46.011502 (+ 1648us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 20:17:46.011506 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:46.011521 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 20:17:46.011587 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:46.012048 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:46.012054 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:46.012060 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 20:17:46.012176 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 20:17:46.012227 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7552,"thread_start_us":8849,"threads_started":1}
I20260809 20:17:46.013163 11355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:45.999322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37927 (local address 127.0.0.1:58854)
0809 20:17:46.007554 (+ 8232us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:46.007596 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:46.007699 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:46.010756 (+ 3057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:46.010762 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:46.010800 (+ 38us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:46.011449 (+ 649us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 20:17:46.011464 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:46.011483 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 20:17:46.011530 (+ 47us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:46.012293 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:46.012298 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:46.012304 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 20:17:46.012706 (+ 402us) client_negotiation.cc:240] Negotiation successful
0809 20:17:46.012941 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8082,"thread_start_us":8796,"threads_started":1}
I20260809 20:17:46.013505 11360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:46.001352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52768 (local address 127.5.61.2:38247)
0809 20:17:46.009177 (+ 7825us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:46.009189 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:46.009229 (+ 40us) server_negotiation.cc:407] Connection header received
0809 20:17:46.009326 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:46.009338 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:46.009621 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:46.010442 (+ 821us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:46.011782 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:46.011791 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:46.011804 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:17:46.011825 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:46.011893 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:17:46.011916 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:46.013054 (+ 1138us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:46.013117 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 20:17:46.013214 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7570,"thread_start_us":7426,"threads_started":1}
I20260809 20:17:46.014174 11358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:46.000014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58854 (local address 127.5.61.1:37927)
0809 20:17:46.010218 (+ 10204us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:46.010223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:46.010255 (+ 32us) server_negotiation.cc:407] Connection header received
0809 20:17:46.010311 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:46.010316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:46.010565 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:46.010757 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:46.011952 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:46.011966 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:46.011980 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:17:46.012045 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:46.012108 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:17:46.012136 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:46.013585 (+ 1449us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:46.013688 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 20:17:46.013784 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10034,"thread_start_us":9901,"threads_started":1}
I20260809 20:17:46.014114 11268 tablet_service.cc:1467] Processing CreateTablet for tablet 3d3a9bd3da0a4518ba37f67122ea6597 (DEFAULT_TABLE table=test-table [id=35129bc478914302bfcbf8ef45875e7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:46.015718 11268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d3a9bd3da0a4518ba37f67122ea6597. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:46.017306 11115 tablet_service.cc:1467] Processing CreateTablet for tablet 3d3a9bd3da0a4518ba37f67122ea6597 (DEFAULT_TABLE table=test-table [id=35129bc478914302bfcbf8ef45875e7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:46.017632 10964 tablet_service.cc:1467] Processing CreateTablet for tablet 3d3a9bd3da0a4518ba37f67122ea6597 (DEFAULT_TABLE table=test-table [id=35129bc478914302bfcbf8ef45875e7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:17:46.018675 11115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d3a9bd3da0a4518ba37f67122ea6597. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:46.018900 10964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d3a9bd3da0a4518ba37f67122ea6597. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:46.030942 11361 tablet_bootstrap.cc:492] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7: Bootstrap starting.
I20260809 20:17:46.032691 11362 tablet_bootstrap.cc:492] T 3d3a9bd3da0a4518ba37f67122ea6597 P 7a241709573d455bb7b88b15600de9d7: Bootstrap starting.
I20260809 20:17:46.035166 11361 tablet_bootstrap.cc:654] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:46.036386 11363 tablet_bootstrap.cc:492] T 3d3a9bd3da0a4518ba37f67122ea6597 P 390d7f48e3a241878524e34d6bfc0c14: Bootstrap starting.
I20260809 20:17:46.036826 11361 log.cc:824] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:46.037030 11362 tablet_bootstrap.cc:654] T 3d3a9bd3da0a4518ba37f67122ea6597 P 7a241709573d455bb7b88b15600de9d7: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:46.038348 11362 log.cc:824] T 3d3a9bd3da0a4518ba37f67122ea6597 P 7a241709573d455bb7b88b15600de9d7: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:46.039901 11363 tablet_bootstrap.cc:654] T 3d3a9bd3da0a4518ba37f67122ea6597 P 390d7f48e3a241878524e34d6bfc0c14: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:46.040400 11361 tablet_bootstrap.cc:492] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7: No bootstrap required, opened a new log
I20260809 20:17:46.040723 11361 ts_tablet_manager.cc:1397] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260809 20:17:46.041075 11363 log.cc:824] T 3d3a9bd3da0a4518ba37f67122ea6597 P 390d7f48e3a241878524e34d6bfc0c14: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:46.041138 11362 tablet_bootstrap.cc:492] T 3d3a9bd3da0a4518ba37f67122ea6597 P 7a241709573d455bb7b88b15600de9d7: No bootstrap required, opened a new log
I20260809 20:17:46.041378 11362 ts_tablet_manager.cc:1397] T 3d3a9bd3da0a4518ba37f67122ea6597 P 7a241709573d455bb7b88b15600de9d7: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 20:17:46.046093 11363 tablet_bootstrap.cc:492] T 3d3a9bd3da0a4518ba37f67122ea6597 P 390d7f48e3a241878524e34d6bfc0c14: No bootstrap required, opened a new log
I20260809 20:17:46.046360 11363 ts_tablet_manager.cc:1397] T 3d3a9bd3da0a4518ba37f67122ea6597 P 390d7f48e3a241878524e34d6bfc0c14: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260809 20:17:46.048910 11362 raft_consensus.cc:348] T 3d3a9bd3da0a4518ba37f67122ea6597 P 7a241709573d455bb7b88b15600de9d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d79746cb96421eb606b1d610db55d7" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40447 } } peers { permanent_uuid: "390d7f48e3a241878524e34d6bfc0c14" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38247 } } peers { permanent_uuid: "7a241709573d455bb7b88b15600de9d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37927 } }
I20260809 20:17:46.049274 11362 raft_consensus.cc:374] T 3d3a9bd3da0a4518ba37f67122ea6597 P 7a241709573d455bb7b88b15600de9d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:46.049396 11362 raft_consensus.cc:729] T 3d3a9bd3da0a4518ba37f67122ea6597 P 7a241709573d455bb7b88b15600de9d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a241709573d455bb7b88b15600de9d7, State: Initialized, Role: FOLLOWER
I20260809 20:17:46.049988 11362 consensus_queue.cc:260] T 3d3a9bd3da0a4518ba37f67122ea6597 P 7a241709573d455bb7b88b15600de9d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d79746cb96421eb606b1d610db55d7" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40447 } } peers { permanent_uuid: "390d7f48e3a241878524e34d6bfc0c14" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38247 } } peers { permanent_uuid: "7a241709573d455bb7b88b15600de9d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37927 } }
I20260809 20:17:46.051048 11037 heartbeater.cc:503] Master 127.5.61.62:41557 was elected leader, sending a full tablet report...
I20260809 20:17:46.051256 11361 raft_consensus.cc:348] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d79746cb96421eb606b1d610db55d7" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40447 } } peers { permanent_uuid: "390d7f48e3a241878524e34d6bfc0c14" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38247 } } peers { permanent_uuid: "7a241709573d455bb7b88b15600de9d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37927 } }
I20260809 20:17:46.051515 11362 ts_tablet_manager.cc:1428] T 3d3a9bd3da0a4518ba37f67122ea6597 P 7a241709573d455bb7b88b15600de9d7: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260809 20:17:46.051769 11361 raft_consensus.cc:374] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:46.051975 11361 raft_consensus.cc:729] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7d79746cb96421eb606b1d610db55d7, State: Initialized, Role: FOLLOWER
I20260809 20:17:46.052551 11361 consensus_queue.cc:260] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d79746cb96421eb606b1d610db55d7" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40447 } } peers { permanent_uuid: "390d7f48e3a241878524e34d6bfc0c14" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38247 } } peers { permanent_uuid: "7a241709573d455bb7b88b15600de9d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37927 } }
I20260809 20:17:46.059674 11342 heartbeater.cc:503] Master 127.5.61.62:41557 was elected leader, sending a full tablet report...
I20260809 20:17:46.059793 11363 raft_consensus.cc:348] T 3d3a9bd3da0a4518ba37f67122ea6597 P 390d7f48e3a241878524e34d6bfc0c14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d79746cb96421eb606b1d610db55d7" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40447 } } peers { permanent_uuid: "390d7f48e3a241878524e34d6bfc0c14" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38247 } } peers { permanent_uuid: "7a241709573d455bb7b88b15600de9d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37927 } }
I20260809 20:17:46.060211 11363 raft_consensus.cc:374] T 3d3a9bd3da0a4518ba37f67122ea6597 P 390d7f48e3a241878524e34d6bfc0c14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:46.060333 11363 raft_consensus.cc:729] T 3d3a9bd3da0a4518ba37f67122ea6597 P 390d7f48e3a241878524e34d6bfc0c14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 390d7f48e3a241878524e34d6bfc0c14, State: Initialized, Role: FOLLOWER
I20260809 20:17:46.060681 11361 ts_tablet_manager.cc:1428] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7: Time spent starting tablet: real 0.020s user 0.018s sys 0.000s
I20260809 20:17:46.060956 11363 consensus_queue.cc:260] T 3d3a9bd3da0a4518ba37f67122ea6597 P 390d7f48e3a241878524e34d6bfc0c14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d79746cb96421eb606b1d610db55d7" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40447 } } peers { permanent_uuid: "390d7f48e3a241878524e34d6bfc0c14" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38247 } } peers { permanent_uuid: "7a241709573d455bb7b88b15600de9d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37927 } }
I20260809 20:17:46.061975 11192 heartbeater.cc:503] Master 127.5.61.62:41557 was elected leader, sending a full tablet report...
I20260809 20:17:46.062577 11363 ts_tablet_manager.cc:1428] T 3d3a9bd3da0a4518ba37f67122ea6597 P 390d7f48e3a241878524e34d6bfc0c14: Time spent starting tablet: real 0.016s user 0.010s sys 0.005s
W20260809 20:17:46.080709 11038 tablet.cc:2369] T 3d3a9bd3da0a4518ba37f67122ea6597 P 7a241709573d455bb7b88b15600de9d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 20:17:46.199935 11343 tablet.cc:2369] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 20:17:46.265898 11193 tablet.cc:2369] T 3d3a9bd3da0a4518ba37f67122ea6597 P 390d7f48e3a241878524e34d6bfc0c14: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:17:46.267292 11369 raft_consensus.cc:482] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:17:46.267627 11369 raft_consensus.cc:504] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d79746cb96421eb606b1d610db55d7" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40447 } } peers { permanent_uuid: "390d7f48e3a241878524e34d6bfc0c14" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38247 } } peers { permanent_uuid: "7a241709573d455bb7b88b15600de9d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37927 } }
I20260809 20:17:46.269390 11369 leader_election.cc:288] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 390d7f48e3a241878524e34d6bfc0c14 (127.5.61.2:38247), 7a241709573d455bb7b88b15600de9d7 (127.5.61.1:37927)
I20260809 20:17:46.272297 11344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:46.269677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:38247 (local address 127.5.61.3:60231)
0809 20:17:46.269973 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:46.269996 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:46.270122 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:46.270588 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:46.270591 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:46.270638 (+ 47us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:46.271392 (+ 754us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 20:17:46.271394 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:46.271409 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 20:17:46.271447 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:46.271816 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:46.271823 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:46.271831 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 20:17:46.271942 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 20:17:46.271990 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159}
I20260809 20:17:46.272300 11360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:46.269900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:60231 (local address 127.5.61.2:38247)
0809 20:17:46.270062 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:46.270068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:46.270090 (+ 22us) server_negotiation.cc:407] Connection header received
0809 20:17:46.270360 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:46.270366 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:46.270434 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:46.270556 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:46.271616 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:46.271624 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:46.271632 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:17:46.271655 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:46.271698 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:17:46.271711 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:46.271822 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:46.271961 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 20:17:46.272025 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 20:17:46.272722 11358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:46.269840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:52217 (local address 127.5.61.1:37927)
0809 20:17:46.270019 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:46.270025 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:46.270647 (+ 622us) server_negotiation.cc:407] Connection header received
0809 20:17:46.270803 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:46.270811 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:46.270891 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:46.270995 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:46.271975 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:46.271981 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:46.271986 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:17:46.272004 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:46.272061 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:17:46.272078 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:46.272193 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:46.272356 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 20:17:46.272441 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 20:17:46.273288 11372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:46.269732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37927 (local address 127.5.61.3:52217)
0809 20:17:46.270529 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:46.270549 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:46.270662 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:46.271042 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:46.271047 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:46.271070 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:46.271727 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 20:17:46.271730 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:46.271741 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 20:17:46.271796 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:46.272166 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:46.272170 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:46.272174 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 20:17:46.272964 (+ 790us) client_negotiation.cc:240] Negotiation successful
0809 20:17:46.273003 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":498,"threads_started":1}
I20260809 20:17:46.273289 11140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d3a9bd3da0a4518ba37f67122ea6597" candidate_uuid: "d7d79746cb96421eb606b1d610db55d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "390d7f48e3a241878524e34d6bfc0c14" is_pre_election: true
I20260809 20:17:46.273900 11140 raft_consensus.cc:2454] T 3d3a9bd3da0a4518ba37f67122ea6597 P 390d7f48e3a241878524e34d6bfc0c14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7d79746cb96421eb606b1d610db55d7 in term 0.
I20260809 20:17:46.274312 10987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d3a9bd3da0a4518ba37f67122ea6597" candidate_uuid: "d7d79746cb96421eb606b1d610db55d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a241709573d455bb7b88b15600de9d7" is_pre_election: true
I20260809 20:17:46.274649 11212 leader_election.cc:302] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 390d7f48e3a241878524e34d6bfc0c14, d7d79746cb96421eb606b1d610db55d7; no voters:
I20260809 20:17:46.274832 10987 raft_consensus.cc:2454] T 3d3a9bd3da0a4518ba37f67122ea6597 P 7a241709573d455bb7b88b15600de9d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7d79746cb96421eb606b1d610db55d7 in term 0.
I20260809 20:17:46.275039 11369 raft_consensus.cc:2781] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:17:46.275190 11369 raft_consensus.cc:482] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:17:46.275312 11369 raft_consensus.cc:3037] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:46.277611 11369 raft_consensus.cc:504] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d79746cb96421eb606b1d610db55d7" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40447 } } peers { permanent_uuid: "390d7f48e3a241878524e34d6bfc0c14" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38247 } } peers { permanent_uuid: "7a241709573d455bb7b88b15600de9d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37927 } }
I20260809 20:17:46.278178 11369 leader_election.cc:288] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7 [CANDIDATE]: Term 1 election: Requested vote from peers 390d7f48e3a241878524e34d6bfc0c14 (127.5.61.2:38247), 7a241709573d455bb7b88b15600de9d7 (127.5.61.1:37927)
I20260809 20:17:46.278642 11140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d3a9bd3da0a4518ba37f67122ea6597" candidate_uuid: "d7d79746cb96421eb606b1d610db55d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "390d7f48e3a241878524e34d6bfc0c14"
I20260809 20:17:46.278684 10987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d3a9bd3da0a4518ba37f67122ea6597" candidate_uuid: "d7d79746cb96421eb606b1d610db55d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a241709573d455bb7b88b15600de9d7"
I20260809 20:17:46.278861 11140 raft_consensus.cc:3037] T 3d3a9bd3da0a4518ba37f67122ea6597 P 390d7f48e3a241878524e34d6bfc0c14 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:46.278872 10987 raft_consensus.cc:3037] T 3d3a9bd3da0a4518ba37f67122ea6597 P 7a241709573d455bb7b88b15600de9d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:46.282135 10987 raft_consensus.cc:2454] T 3d3a9bd3da0a4518ba37f67122ea6597 P 7a241709573d455bb7b88b15600de9d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7d79746cb96421eb606b1d610db55d7 in term 1.
I20260809 20:17:46.282130 11140 raft_consensus.cc:2454] T 3d3a9bd3da0a4518ba37f67122ea6597 P 390d7f48e3a241878524e34d6bfc0c14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7d79746cb96421eb606b1d610db55d7 in term 1.
I20260809 20:17:46.282565 11213 leader_election.cc:302] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a241709573d455bb7b88b15600de9d7, d7d79746cb96421eb606b1d610db55d7; no voters:
I20260809 20:17:46.282851 11369 raft_consensus.cc:2781] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:46.283284 11369 raft_consensus.cc:686] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7 [term 1 LEADER]: Becoming Leader. State: Replica: d7d79746cb96421eb606b1d610db55d7, State: Running, Role: LEADER
I20260809 20:17:46.283839 11369 consensus_queue.cc:237] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d79746cb96421eb606b1d610db55d7" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40447 } } peers { permanent_uuid: "390d7f48e3a241878524e34d6bfc0c14" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38247 } } peers { permanent_uuid: "7a241709573d455bb7b88b15600de9d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37927 } }
I20260809 20:17:46.288326 10826 catalog_manager.cc:5495] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7d79746cb96421eb606b1d610db55d7 (127.5.61.3). New cstate: current_term: 1 leader_uuid: "d7d79746cb96421eb606b1d610db55d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d79746cb96421eb606b1d610db55d7" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "390d7f48e3a241878524e34d6bfc0c14" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a241709573d455bb7b88b15600de9d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37927 } health_report { overall_health: UNKNOWN } } }
I20260809 20:17:46.305361 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:46.303032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48634 (local address 127.5.61.3:40447)
0809 20:17:46.303212 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:46.303218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:46.303239 (+ 21us) server_negotiation.cc:407] Connection header received
0809 20:17:46.303438 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:46.303442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:46.303519 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:46.303670 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:46.304700 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:46.304706 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:46.304711 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:17:46.304737 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:46.304771 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:17:46.304783 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:46.304927 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:46.305018 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 20:17:46.305084 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 20:17:46.313522 11140 raft_consensus.cc:1261] T 3d3a9bd3da0a4518ba37f67122ea6597 P 390d7f48e3a241878524e34d6bfc0c14 [term 1 FOLLOWER]: Refusing update from remote peer d7d79746cb96421eb606b1d610db55d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:17:46.313520 10987 raft_consensus.cc:1261] T 3d3a9bd3da0a4518ba37f67122ea6597 P 7a241709573d455bb7b88b15600de9d7 [term 1 FOLLOWER]: Refusing update from remote peer d7d79746cb96421eb606b1d610db55d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:17:46.314486 11369 consensus_queue.cc:1035] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "390d7f48e3a241878524e34d6bfc0c14" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:17:46.314787 11375 consensus_queue.cc:1035] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a241709573d455bb7b88b15600de9d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:17:46.347774 10826 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:17:46.348079 10826 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:17:46.351847 10826 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6850f2c04c0041c4b3c7ccbee16efe02: Sending DeleteTablet for 3 replicas of tablet 3d3a9bd3da0a4518ba37f67122ea6597
I20260809 20:17:46.353075 11115 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d3a9bd3da0a4518ba37f67122ea6597 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:17:46 UTC) from {username='slave'} at 127.0.0.1:52768
I20260809 20:17:46.353204 10964 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d3a9bd3da0a4518ba37f67122ea6597 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:17:46 UTC) from {username='slave'} at 127.0.0.1:58854
I20260809 20:17:46.353228 11268 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d3a9bd3da0a4518ba37f67122ea6597 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:17:46 UTC) from {username='slave'} at 127.0.0.1:48624
I20260809 20:17:46.354908 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 10891
I20260809 20:17:46.365532 11391 tablet_replica.cc:330] T 3d3a9bd3da0a4518ba37f67122ea6597 P 7a241709573d455bb7b88b15600de9d7: stopping tablet replica
I20260809 20:17:46.366282 11389 tablet_replica.cc:330] T 3d3a9bd3da0a4518ba37f67122ea6597 P 390d7f48e3a241878524e34d6bfc0c14: stopping tablet replica
I20260809 20:17:46.367074 11389 raft_consensus.cc:2229] T 3d3a9bd3da0a4518ba37f67122ea6597 P 390d7f48e3a241878524e34d6bfc0c14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:17:46.367983 11391 raft_consensus.cc:2229] T 3d3a9bd3da0a4518ba37f67122ea6597 P 7a241709573d455bb7b88b15600de9d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:17:46.368665 11391 raft_consensus.cc:2258] T 3d3a9bd3da0a4518ba37f67122ea6597 P 7a241709573d455bb7b88b15600de9d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:46.369235 11389 raft_consensus.cc:2258] T 3d3a9bd3da0a4518ba37f67122ea6597 P 390d7f48e3a241878524e34d6bfc0c14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:46.371240 11358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:46.357020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58866 (local address 127.5.61.1:37927)
0809 20:17:46.357257 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:46.357267 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:46.368287 (+ 11020us) server_negotiation.cc:407] Connection header received
0809 20:17:46.368788 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:46.368795 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:46.368904 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:46.368989 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:46.370332 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:46.370340 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:46.370345 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:17:46.370369 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:46.370408 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:17:46.370427 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:46.370533 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:46.370840 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 20:17:46.370905 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 20:17:46.372568 11031 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:46.372991 11390 tablet_replica.cc:330] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7: stopping tablet replica
I20260809 20:17:46.373406 11389 ts_tablet_manager.cc:1905] T 3d3a9bd3da0a4518ba37f67122ea6597 P 390d7f48e3a241878524e34d6bfc0c14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:46.373431 11390 raft_consensus.cc:2229] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:17:46.374213 11390 raft_consensus.cc:2258] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:46.376250 11390 ts_tablet_manager.cc:1905] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:46.381001 11389 ts_tablet_manager.cc:1918] T 3d3a9bd3da0a4518ba37f67122ea6597 P 390d7f48e3a241878524e34d6bfc0c14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:17:46.381279 11389 log.cc:1196] T 3d3a9bd3da0a4518ba37f67122ea6597 P 390d7f48e3a241878524e34d6bfc0c14: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/3d3a9bd3da0a4518ba37f67122ea6597
I20260809 20:17:46.381960 11389 ts_tablet_manager.cc:1939] T 3d3a9bd3da0a4518ba37f67122ea6597 P 390d7f48e3a241878524e34d6bfc0c14: Deleting consensus metadata
I20260809 20:17:46.384057 10806 catalog_manager.cc:4841] TS 390d7f48e3a241878524e34d6bfc0c14 (127.5.61.2:38247): tablet 3d3a9bd3da0a4518ba37f67122ea6597 (table test-table [id=35129bc478914302bfcbf8ef45875e7a]) successfully deleted
I20260809 20:17:46.384678 11390 ts_tablet_manager.cc:1918] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:17:46.385017 11390 log.cc:1196] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/3d3a9bd3da0a4518ba37f67122ea6597
I20260809 20:17:46.386065 11390 ts_tablet_manager.cc:1939] T 3d3a9bd3da0a4518ba37f67122ea6597 P d7d79746cb96421eb606b1d610db55d7: Deleting consensus metadata
I20260809 20:17:46.388196 10806 catalog_manager.cc:4841] TS d7d79746cb96421eb606b1d610db55d7 (127.5.61.3:40447): tablet 3d3a9bd3da0a4518ba37f67122ea6597 (table test-table [id=35129bc478914302bfcbf8ef45875e7a]) successfully deleted
I20260809 20:17:46.373584 11391 ts_tablet_manager.cc:1905] T 3d3a9bd3da0a4518ba37f67122ea6597 P 7a241709573d455bb7b88b15600de9d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:46.473024 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 10891
I20260809 20:17:46.493129 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 11041
W20260809 20:17:46.493119 10807 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.61.1:37927 (error 108)
I20260809 20:17:46.495908 11355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:46.495243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37927 (local address 127.0.0.1:58878)
0809 20:17:46.495450 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:46.495684 (+ 234us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.1:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260809 20:17:46.496168 10807 catalog_manager.cc:4584] TS 7a241709573d455bb7b88b15600de9d7 (127.5.61.1:37927): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3d3a9bd3da0a4518ba37f67122ea6597: Network error: Client connection negotiation failed: client connection to 127.5.61.1:37927: connect: Connection refused (error 111)
I20260809 20:17:46.497066 11360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:46.494290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52776 (local address 127.5.61.2:38247)
0809 20:17:46.494479 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:46.494484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:46.494591 (+ 107us) server_negotiation.cc:407] Connection header received
0809 20:17:46.494760 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:46.494766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:46.494884 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:46.495001 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:46.496196 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:46.496203 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:46.496208 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:17:46.496235 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:46.496273 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:17:46.496310 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:46.496446 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:46.496690 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 20:17:46.496755 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 20:17:46.497854 11187 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:46.550186 11355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:46.549590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37927 (local address 127.0.0.1:58888)
0809 20:17:46.549824 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:46.549985 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.1:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 20:17:46.551414 11355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:46.550950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37927 (local address 127.0.0.1:58896)
0809 20:17:46.551179 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:46.551274 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.1:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 20:17:46.596457 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 11041
I20260809 20:17:46.611855 11355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:46.611394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37927 (local address 127.0.0.1:58908)
0809 20:17:46.611622 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:46.611715 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.1:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 20:17:46.613480 11355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:46.612841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37927 (local address 127.0.0.1:58914)
0809 20:17:46.613163 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:46.613300 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.1:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222}
I20260809 20:17:46.624651 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 11195
I20260809 20:17:46.627681 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:46.625476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48636 (local address 127.5.61.3:40447)
0809 20:17:46.625669 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:46.625675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:46.625700 (+ 25us) server_negotiation.cc:407] Connection header received
0809 20:17:46.625794 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:46.625800 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:46.625887 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:46.626016 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:46.626936 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:46.626942 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:46.626948 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:17:46.626975 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:46.627015 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:17:46.627030 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:46.627188 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:46.627270 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 20:17:46.627330 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 20:17:46.628399 11336 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:46.694548 11355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:46.693929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37927 (local address 127.0.0.1:58928)
0809 20:17:46.694200 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:46.694328 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.1:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260809 20:17:46.695816 11355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:46.695270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37927 (local address 127.0.0.1:58944)
0809 20:17:46.695470 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:46.695552 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.1:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260809 20:17:46.747699 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 11195
I20260809 20:17:46.771425 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 10786
I20260809 20:17:46.772680 10859 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:46.878490 11355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:46.877493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37927 (local address 127.0.0.1:58956)
0809 20:17:46.877751 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:46.877916 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.1:37927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260809 20:17:46.878515 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 10786
2026-08-09T20:17:46Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2800 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 20:17:46.924084 5364 mini_oidc.cc:152] Starting JWKS server
I20260809 20:17:46.926818 5364 webserver.cc:460] Webserver started at https://127.0.0.1:34147/ using document root <none> and password file <none>
I20260809 20:17:46.927220 5364 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 20:17:46.927909 5364 webserver.cc:460] Webserver started at http://127.0.0.1:33731/ using document root <none> and password file <none>
2026-08-09T20:17:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:17:46Z Disabled control of system clock
I20260809 20:17:46.957463 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:37787
--webserver_interface=127.5.61.62
--webserver_port=0
--builtin_ntp_servers=127.5.61.20:45089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:37787
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34147/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786306623011809-5364-0/testchainca.cert with env {}
W20260809 20:17:47.293682 11407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:47.294072 11407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:47.294193 11407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:47.301246 11407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:17:47.301349 11407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:47.301401 11407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:47.301463 11407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:17:47.301512 11407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 20:17:47.301579 11407 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 20:17:47.301625 11407 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34147/jwks.json
I20260809 20:17:47.312304 11407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:45089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:37787
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:37787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34147/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786306623011809-5364-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:47.313530 11407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:47.314811 11407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:47.326316 11418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:47.326390 11413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:47.326407 11416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:47.327210 11407 server_base.cc:1034] running on GCE node
I20260809 20:17:47.328024 11407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:47.329347 11407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:47.330579 11407 hybrid_clock.cc:648] HybridClock initialized: now 1786306667330548 us; error 45 us; skew 500 ppm
I20260809 20:17:47.330920 11407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:17:47.333182 11407 webserver.cc:460] Webserver started at http://127.5.61.62:46007/ using document root <none> and password file <none>
I20260809 20:17:47.333853 11407 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:47.333990 11407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:47.334309 11407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:47.337162 11407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "f4188279ee0f48eea10f6c57d676591e"
format_stamp: "Formatted at 2026-08-09 20:17:47 on dist-test-slave-6k22"
server_key: "f04a554ad857f921f5a95f1e8dfc4f02"
server_key_iv: "82e087ecad766992932bfd0ff9f65aa9"
server_key_version: "encryptionkey@0"
I20260809 20:17:47.337998 11407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "f4188279ee0f48eea10f6c57d676591e"
format_stamp: "Formatted at 2026-08-09 20:17:47 on dist-test-slave-6k22"
server_key: "f04a554ad857f921f5a95f1e8dfc4f02"
server_key_iv: "82e087ecad766992932bfd0ff9f65aa9"
server_key_version: "encryptionkey@0"
I20260809 20:17:47.342862 11407 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 20:17:47.346169 11425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:47.347500 11407 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 20:17:47.347724 11407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "f4188279ee0f48eea10f6c57d676591e"
format_stamp: "Formatted at 2026-08-09 20:17:47 on dist-test-slave-6k22"
server_key: "f04a554ad857f921f5a95f1e8dfc4f02"
server_key_iv: "82e087ecad766992932bfd0ff9f65aa9"
server_key_version: "encryptionkey@0"
I20260809 20:17:47.347939 11407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:47.359267 11407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:47.360041 11407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:47.360319 11407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:47.378075 11407 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:37787
I20260809 20:17:47.378094 11482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:37787 every 8 connection(s)
I20260809 20:17:47.379416 11407 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:17:47.383116 11483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:47.385692 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 11407
I20260809 20:17:47.385929 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:17:47.386711 5364 external_mini_cluster.cc:1428] Setting key da607f60f27dd30bdf837534a7d66528
I20260809 20:17:47.395676 11483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4188279ee0f48eea10f6c57d676591e: Bootstrap starting.
I20260809 20:17:47.399433 11483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4188279ee0f48eea10f6c57d676591e: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:47.400574 11483 log.cc:824] T 00000000000000000000000000000000 P f4188279ee0f48eea10f6c57d676591e: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:47.400897 11489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:47.388727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44514 (local address 127.5.61.62:37787)
0809 20:17:47.389719 (+ 992us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:47.389729 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:47.389770 (+ 41us) server_negotiation.cc:407] Connection header received
0809 20:17:47.391407 (+ 1637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:47.391430 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:47.391960 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:47.392302 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:47.393017 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:47.394303 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:47.396204 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:47.396834 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:47.396958 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:47.398409 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:47.398416 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:47.398429 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:17:47.398952 (+ 523us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:47.399037 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:17:47.399056 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:47.399155 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:47.399448 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 20:17:47.399855 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":378,"threads_started":1}
I20260809 20:17:47.404037 11483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4188279ee0f48eea10f6c57d676591e: No bootstrap required, opened a new log
I20260809 20:17:47.405351 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:17:47.412218 11483 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4188279ee0f48eea10f6c57d676591e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4188279ee0f48eea10f6c57d676591e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37787 } }
I20260809 20:17:47.412588 11483 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4188279ee0f48eea10f6c57d676591e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:47.412708 11483 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4188279ee0f48eea10f6c57d676591e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4188279ee0f48eea10f6c57d676591e, State: Initialized, Role: FOLLOWER
I20260809 20:17:47.413376 11483 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4188279ee0f48eea10f6c57d676591e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4188279ee0f48eea10f6c57d676591e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37787 } }
I20260809 20:17:47.413691 11483 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4188279ee0f48eea10f6c57d676591e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:17:47.413856 11483 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4188279ee0f48eea10f6c57d676591e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:17:47.414049 11483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4188279ee0f48eea10f6c57d676591e [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:47.416458 11483 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4188279ee0f48eea10f6c57d676591e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4188279ee0f48eea10f6c57d676591e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37787 } }
I20260809 20:17:47.416973 11483 leader_election.cc:302] T 00000000000000000000000000000000 P f4188279ee0f48eea10f6c57d676591e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4188279ee0f48eea10f6c57d676591e; no voters:
I20260809 20:17:47.417625 11483 leader_election.cc:288] T 00000000000000000000000000000000 P f4188279ee0f48eea10f6c57d676591e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:17:47.417801 11491 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4188279ee0f48eea10f6c57d676591e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:47.418699 11491 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4188279ee0f48eea10f6c57d676591e [term 1 LEADER]: Becoming Leader. State: Replica: f4188279ee0f48eea10f6c57d676591e, State: Running, Role: LEADER
I20260809 20:17:47.419184 11491 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4188279ee0f48eea10f6c57d676591e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4188279ee0f48eea10f6c57d676591e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37787 } }
I20260809 20:17:47.421034 11492 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4188279ee0f48eea10f6c57d676591e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4188279ee0f48eea10f6c57d676591e. Latest consensus state: current_term: 1 leader_uuid: "f4188279ee0f48eea10f6c57d676591e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4188279ee0f48eea10f6c57d676591e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37787 } } }
I20260809 20:17:47.421440 11483 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4188279ee0f48eea10f6c57d676591e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:17:47.421566 11492 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4188279ee0f48eea10f6c57d676591e [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:47.422016 11493 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4188279ee0f48eea10f6c57d676591e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4188279ee0f48eea10f6c57d676591e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4188279ee0f48eea10f6c57d676591e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37787 } } }
I20260809 20:17:47.422287 11493 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4188279ee0f48eea10f6c57d676591e [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:47.423745 11495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:17:47.430012 11495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:17:47.438220 11495 catalog_manager.cc:1347] Generated new cluster ID: 6ae49c498a6e4708aad6640a5d96a374
I20260809 20:17:47.438325 11495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:17:47.452903 11495 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:17:47.454553 11495 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:17:47.467352 11495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4188279ee0f48eea10f6c57d676591e: Generated new TSK 0
I20260809 20:17:47.468204 11495 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 20:17:47.511644 11521 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 20:17:47.511883 11521 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 20:17:47.512982 11489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:47.491332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44526 (local address 127.5.61.62:37787)
0809 20:17:47.491527 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:47.491534 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:47.492142 (+ 608us) server_negotiation.cc:407] Connection header received
0809 20:17:47.492451 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:47.492457 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:47.492539 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:47.492685 (+ 146us) server_negotiation.cc:226] Negotiated authn=JWT
0809 20:17:47.493177 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:47.494294 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:47.495647 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:47.496362 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:47.496483 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:47.496973 (+ 490us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 20:17:47.512348 (+ 15375us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34147/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 20:17:47.512694 (+ 346us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44526: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34147/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 20:17:47.513098 11489 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44526: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34147/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 20:17:47.514504 11520 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:17:47.491333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:37787 (local address 127.0.0.1:44526)
0809 20:17:47.492043 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:47.492060 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:47.492198 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:47.492722 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:47.492725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:47.492735 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0809 20:17:47.493015 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:47.493022 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:47.494431 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:47.494435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:47.495489 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:47.495500 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:47.496491 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:47.496529 (+ 38us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:47.496662 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:47.496716 (+ 54us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 20:17:47.512602 (+ 15886us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34147/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 20:17:47.512831 (+ 229us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.61.62:37787: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34147/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":377,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.5.61.62:37787: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34147/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 20:17:47.516705 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 11407
I20260809 20:17:47.517807 11476 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:47.608983 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 11407
2026-08-09T20:17:47Z chronyd exiting
I20260809 20:17:47.662250 5364 test_util.cc:175] -----------------------------------------------
I20260809 20:17:47.662385 5364 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786306623011809-5364-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (742 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 20:17:47.666290 5364 mini_oidc.cc:152] Starting JWKS server
I20260809 20:17:47.668179 5364 webserver.cc:460] Webserver started at https://127.0.0.1:35697/ using document root <none> and password file <none>
I20260809 20:17:47.668413 5364 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 20:17:47.669020 5364 webserver.cc:460] Webserver started at http://127.0.0.1:35233/ using document root <none> and password file <none>
2026-08-09T20:17:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:17:47Z Disabled control of system clock
I20260809 20:17:47.698928 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:34811
--webserver_interface=127.5.61.62
--webserver_port=0
--builtin_ntp_servers=127.5.61.20:43351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:34811
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35697/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786306623011809-5364-0/testchainca.cert with env {}
W20260809 20:17:48.034049 11530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:48.034439 11530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:48.034616 11530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:48.041621 11530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:17:48.041766 11530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:48.041834 11530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:48.041916 11530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:17:48.042008 11530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 20:17:48.042098 11530 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 20:17:48.042157 11530 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35697/jwks.json
I20260809 20:17:48.051904 11530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:43351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:34811
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:34811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35697/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786306623011809-5364-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:48.053201 11530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:48.054471 11530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:48.065975 11535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:48.066030 11538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:48.066080 11536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:48.066967 11530 server_base.cc:1034] running on GCE node
I20260809 20:17:48.067862 11530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:48.069149 11530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:48.070363 11530 hybrid_clock.cc:648] HybridClock initialized: now 1786306668070313 us; error 50 us; skew 500 ppm
I20260809 20:17:48.070750 11530 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:17:48.072978 11530 webserver.cc:460] Webserver started at http://127.5.61.62:46327/ using document root <none> and password file <none>
I20260809 20:17:48.073657 11530 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:48.073794 11530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:48.074169 11530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:48.077014 11530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "5804ed01c1754689b8dc84df0beb6eab"
format_stamp: "Formatted at 2026-08-09 20:17:48 on dist-test-slave-6k22"
server_key: "d944ba314c2ac8206b9f3753240788fe"
server_key_iv: "9cc6d6b70a4e809071e524c6dbb74c48"
server_key_version: "encryptionkey@0"
I20260809 20:17:48.077800 11530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "5804ed01c1754689b8dc84df0beb6eab"
format_stamp: "Formatted at 2026-08-09 20:17:48 on dist-test-slave-6k22"
server_key: "d944ba314c2ac8206b9f3753240788fe"
server_key_iv: "9cc6d6b70a4e809071e524c6dbb74c48"
server_key_version: "encryptionkey@0"
I20260809 20:17:48.082877 11530 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 20:17:48.086153 11544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:48.087657 11530 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:17:48.087833 11530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "5804ed01c1754689b8dc84df0beb6eab"
format_stamp: "Formatted at 2026-08-09 20:17:48 on dist-test-slave-6k22"
server_key: "d944ba314c2ac8206b9f3753240788fe"
server_key_iv: "9cc6d6b70a4e809071e524c6dbb74c48"
server_key_version: "encryptionkey@0"
I20260809 20:17:48.088043 11530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:48.124055 11530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:48.124914 11530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:48.125219 11530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:48.143529 11530 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:34811
I20260809 20:17:48.143716 11604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:34811 every 8 connection(s)
I20260809 20:17:48.144918 11530 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:17:48.147913 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 11530
I20260809 20:17:48.148118 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:17:48.148643 11605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:48.148826 5364 external_mini_cluster.cc:1428] Setting key f36e901b6600e20a41b51d790e2da2d4
I20260809 20:17:48.162489 11605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5804ed01c1754689b8dc84df0beb6eab: Bootstrap starting.
I20260809 20:17:48.162770 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:48.150777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50680 (local address 127.5.61.62:34811)
0809 20:17:48.151902 (+ 1125us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:48.151912 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:48.151958 (+ 46us) server_negotiation.cc:407] Connection header received
0809 20:17:48.153470 (+ 1512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:48.153490 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:48.153938 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:48.154247 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:48.154851 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:48.156209 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:48.158026 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:48.158713 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:48.158852 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:48.160185 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:48.160192 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:48.160208 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:17:48.160707 (+ 499us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:48.160790 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:17:48.160813 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:48.160906 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:48.161200 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 20:17:48.161538 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":423,"threads_started":1}
I20260809 20:17:48.165848 11605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5804ed01c1754689b8dc84df0beb6eab: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:48.166435 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:17:48.166998 11605 log.cc:824] T 00000000000000000000000000000000 P 5804ed01c1754689b8dc84df0beb6eab: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:48.169843 11605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5804ed01c1754689b8dc84df0beb6eab: No bootstrap required, opened a new log
I20260809 20:17:48.179471 11605 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5804ed01c1754689b8dc84df0beb6eab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5804ed01c1754689b8dc84df0beb6eab" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 34811 } }
I20260809 20:17:48.179898 11605 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5804ed01c1754689b8dc84df0beb6eab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:48.180020 11605 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5804ed01c1754689b8dc84df0beb6eab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5804ed01c1754689b8dc84df0beb6eab, State: Initialized, Role: FOLLOWER
I20260809 20:17:48.180689 11605 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5804ed01c1754689b8dc84df0beb6eab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5804ed01c1754689b8dc84df0beb6eab" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 34811 } }
I20260809 20:17:48.180980 11605 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5804ed01c1754689b8dc84df0beb6eab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:17:48.181123 11605 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5804ed01c1754689b8dc84df0beb6eab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:17:48.181277 11605 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5804ed01c1754689b8dc84df0beb6eab [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:48.183314 11605 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5804ed01c1754689b8dc84df0beb6eab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5804ed01c1754689b8dc84df0beb6eab" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 34811 } }
I20260809 20:17:48.183820 11605 leader_election.cc:302] T 00000000000000000000000000000000 P 5804ed01c1754689b8dc84df0beb6eab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5804ed01c1754689b8dc84df0beb6eab; no voters:
I20260809 20:17:48.184371 11605 leader_election.cc:288] T 00000000000000000000000000000000 P 5804ed01c1754689b8dc84df0beb6eab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:17:48.184476 11611 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5804ed01c1754689b8dc84df0beb6eab [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:48.185515 11611 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5804ed01c1754689b8dc84df0beb6eab [term 1 LEADER]: Becoming Leader. State: Replica: 5804ed01c1754689b8dc84df0beb6eab, State: Running, Role: LEADER
I20260809 20:17:48.186028 11611 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5804ed01c1754689b8dc84df0beb6eab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5804ed01c1754689b8dc84df0beb6eab" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 34811 } }
I20260809 20:17:48.186489 11605 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5804ed01c1754689b8dc84df0beb6eab [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:17:48.189064 11613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5804ed01c1754689b8dc84df0beb6eab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5804ed01c1754689b8dc84df0beb6eab. Latest consensus state: current_term: 1 leader_uuid: "5804ed01c1754689b8dc84df0beb6eab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5804ed01c1754689b8dc84df0beb6eab" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 34811 } } }
I20260809 20:17:48.189098 11612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5804ed01c1754689b8dc84df0beb6eab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5804ed01c1754689b8dc84df0beb6eab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5804ed01c1754689b8dc84df0beb6eab" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 34811 } } }
I20260809 20:17:48.189467 11613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5804ed01c1754689b8dc84df0beb6eab [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:48.189523 11612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5804ed01c1754689b8dc84df0beb6eab [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:48.191211 11620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:17:48.196323 11620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:17:48.204329 11620 catalog_manager.cc:1347] Generated new cluster ID: c653a1ca6c1c4142bd4a071a478609b4
I20260809 20:17:48.204438 11620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:17:48.223412 11620 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:17:48.224700 11620 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:17:48.231433 11620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5804ed01c1754689b8dc84df0beb6eab: Generated new TSK 0
I20260809 20:17:48.232190 11620 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 20:17:48.272585 11636 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 20:17:48.272704 11636 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 20:17:48.273609 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:48.253613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50692 (local address 127.5.61.62:34811)
0809 20:17:48.253900 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:48.253907 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:48.254304 (+ 397us) server_negotiation.cc:407] Connection header received
0809 20:17:48.254528 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:48.254532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:48.254601 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:48.254721 (+ 120us) server_negotiation.cc:226] Negotiated authn=JWT
0809 20:17:48.255284 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:48.256081 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:48.258066 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:48.258509 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:48.258601 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:48.259214 (+ 613us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 20:17:48.273152 (+ 13938us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35697/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 20:17:48.273399 (+ 247us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50692: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35697/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":74}
W20260809 20:17:48.273761 11609 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50692: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35697/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 20:17:48.274012 11635 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:17:48.253487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:34811 (local address 127.0.0.1:50692)
0809 20:17:48.254192 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:48.254216 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:48.254362 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:48.254782 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:48.254788 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:48.254801 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0809 20:17:48.255102 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:48.255114 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:48.256215 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:48.256222 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:48.257913 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:48.257927 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:48.258632 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:48.258699 (+ 67us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:48.258916 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:48.258965 (+ 49us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 20:17:48.273356 (+ 14391us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35697/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 20:17:48.273599 (+ 243us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.61.62:34811: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35697/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":384,"threads_started":1}
I20260809 20:17:48.275382 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 11530
I20260809 20:17:48.276613 11599 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:48.368709 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 11530
2026-08-09T20:17:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (761 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 20:17:48.427500 5364 mini_oidc.cc:152] Starting JWKS server
I20260809 20:17:48.429466 5364 webserver.cc:460] Webserver started at https://127.0.0.1:46031/ using document root <none> and password file <none>
I20260809 20:17:48.429728 5364 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 20:17:48.430377 5364 webserver.cc:460] Webserver started at http://127.0.0.1:45989/ using document root <none> and password file <none>
2026-08-09T20:17:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:17:48Z Disabled control of system clock
I20260809 20:17:48.461632 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:38327
--webserver_interface=127.5.61.62
--webserver_port=0
--builtin_ntp_servers=127.5.61.20:34825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:38327
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46031/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 20:17:48.786784 11645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:48.787078 11645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:48.787253 11645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:48.794409 11645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:17:48.794548 11645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:48.794644 11645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:48.794755 11645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:17:48.794836 11645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 20:17:48.794938 11645 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 20:17:48.794999 11645 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46031/jwks.json
I20260809 20:17:48.804211 11645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:34825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:38327
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:38327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46031/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:48.805472 11645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:48.807274 11645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:48.819041 11655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:48.819074 11653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:48.819772 11652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:48.820189 11645 server_base.cc:1034] running on GCE node
I20260809 20:17:48.820995 11645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:48.822342 11645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:48.823544 11645 hybrid_clock.cc:648] HybridClock initialized: now 1786306668823512 us; error 33 us; skew 500 ppm
I20260809 20:17:48.823946 11645 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:17:48.826131 11645 webserver.cc:460] Webserver started at http://127.5.61.62:43877/ using document root <none> and password file <none>
I20260809 20:17:48.826818 11645 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:48.826937 11645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:48.827306 11645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:48.830310 11645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "ec3e6cadb3384ccfad2d6227c72f9390"
format_stamp: "Formatted at 2026-08-09 20:17:48 on dist-test-slave-6k22"
server_key: "5ef51b8b7b79c8821613dbe63321b0c8"
server_key_iv: "6092170fee6add1318e4d7cdfedb2b95"
server_key_version: "encryptionkey@0"
I20260809 20:17:48.831095 11645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "ec3e6cadb3384ccfad2d6227c72f9390"
format_stamp: "Formatted at 2026-08-09 20:17:48 on dist-test-slave-6k22"
server_key: "5ef51b8b7b79c8821613dbe63321b0c8"
server_key_iv: "6092170fee6add1318e4d7cdfedb2b95"
server_key_version: "encryptionkey@0"
I20260809 20:17:48.836098 11645 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 20:17:48.839372 11661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:48.840843 11645 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:17:48.841043 11645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "ec3e6cadb3384ccfad2d6227c72f9390"
format_stamp: "Formatted at 2026-08-09 20:17:48 on dist-test-slave-6k22"
server_key: "5ef51b8b7b79c8821613dbe63321b0c8"
server_key_iv: "6092170fee6add1318e4d7cdfedb2b95"
server_key_version: "encryptionkey@0"
I20260809 20:17:48.841279 11645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:48.855621 11645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:48.856407 11645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:48.856686 11645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:48.875075 11645 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:38327
I20260809 20:17:48.875095 11720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:38327 every 8 connection(s)
I20260809 20:17:48.876338 11645 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:17:48.879865 11723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:48.881034 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 11645
I20260809 20:17:48.881270 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:17:48.881907 5364 external_mini_cluster.cc:1428] Setting key 74df31a15153e2a83c39f1cc190b9ae2
I20260809 20:17:48.893010 11723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec3e6cadb3384ccfad2d6227c72f9390: Bootstrap starting.
I20260809 20:17:48.894973 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:48.883715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33512 (local address 127.5.61.62:38327)
0809 20:17:48.884708 (+ 993us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:48.884717 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:48.884764 (+ 47us) server_negotiation.cc:407] Connection header received
0809 20:17:48.886328 (+ 1564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:48.886351 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:48.886780 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:48.887081 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:48.887671 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:48.888907 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:48.890760 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:48.891396 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:48.891517 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:48.892771 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:48.892778 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:48.892789 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:17:48.893371 (+ 582us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:48.893445 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:17:48.893465 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:48.893555 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:48.893790 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 20:17:48.894093 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":375,"threads_started":1}
I20260809 20:17:48.896504 11723 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec3e6cadb3384ccfad2d6227c72f9390: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:48.897679 11723 log.cc:824] T 00000000000000000000000000000000 P ec3e6cadb3384ccfad2d6227c72f9390: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:48.898842 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:17:48.900985 11723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec3e6cadb3384ccfad2d6227c72f9390: No bootstrap required, opened a new log
I20260809 20:17:48.908432 11723 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec3e6cadb3384ccfad2d6227c72f9390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec3e6cadb3384ccfad2d6227c72f9390" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38327 } }
I20260809 20:17:48.908860 11723 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec3e6cadb3384ccfad2d6227c72f9390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:48.908991 11723 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec3e6cadb3384ccfad2d6227c72f9390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3e6cadb3384ccfad2d6227c72f9390, State: Initialized, Role: FOLLOWER
I20260809 20:17:48.909605 11723 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec3e6cadb3384ccfad2d6227c72f9390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec3e6cadb3384ccfad2d6227c72f9390" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38327 } }
I20260809 20:17:48.909845 11723 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec3e6cadb3384ccfad2d6227c72f9390 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:17:48.909994 11723 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec3e6cadb3384ccfad2d6227c72f9390 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:17:48.910173 11723 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec3e6cadb3384ccfad2d6227c72f9390 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:48.912367 11723 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec3e6cadb3384ccfad2d6227c72f9390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec3e6cadb3384ccfad2d6227c72f9390" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38327 } }
I20260809 20:17:48.912882 11723 leader_election.cc:302] T 00000000000000000000000000000000 P ec3e6cadb3384ccfad2d6227c72f9390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec3e6cadb3384ccfad2d6227c72f9390; no voters:
I20260809 20:17:48.913480 11723 leader_election.cc:288] T 00000000000000000000000000000000 P ec3e6cadb3384ccfad2d6227c72f9390 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:17:48.913657 11728 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec3e6cadb3384ccfad2d6227c72f9390 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:48.914397 11728 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec3e6cadb3384ccfad2d6227c72f9390 [term 1 LEADER]: Becoming Leader. State: Replica: ec3e6cadb3384ccfad2d6227c72f9390, State: Running, Role: LEADER
I20260809 20:17:48.915125 11728 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec3e6cadb3384ccfad2d6227c72f9390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec3e6cadb3384ccfad2d6227c72f9390" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38327 } }
I20260809 20:17:48.915800 11723 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec3e6cadb3384ccfad2d6227c72f9390 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:17:48.918402 11729 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec3e6cadb3384ccfad2d6227c72f9390 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec3e6cadb3384ccfad2d6227c72f9390" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec3e6cadb3384ccfad2d6227c72f9390" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38327 } } }
I20260809 20:17:48.918792 11729 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec3e6cadb3384ccfad2d6227c72f9390 [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:48.918514 11730 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec3e6cadb3384ccfad2d6227c72f9390 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec3e6cadb3384ccfad2d6227c72f9390. Latest consensus state: current_term: 1 leader_uuid: "ec3e6cadb3384ccfad2d6227c72f9390" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec3e6cadb3384ccfad2d6227c72f9390" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38327 } } }
I20260809 20:17:48.919167 11730 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec3e6cadb3384ccfad2d6227c72f9390 [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:48.924690 11736 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:17:48.929720 11736 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:17:48.937805 11736 catalog_manager.cc:1347] Generated new cluster ID: fc0f9cc02dc84ca2bec757435d48ab4b
I20260809 20:17:48.937939 11736 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:17:48.952565 11736 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:17:48.954206 11736 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:17:48.966246 11736 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec3e6cadb3384ccfad2d6227c72f9390: Generated new TSK 0
I20260809 20:17:48.967198 11736 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:17:48.989389 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:48.980931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33520 (local address 127.5.61.62:38327)
0809 20:17:48.981121 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:48.981127 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:48.981549 (+ 422us) server_negotiation.cc:407] Connection header received
0809 20:17:48.981774 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:48.981780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:48.981877 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:48.982033 (+ 156us) server_negotiation.cc:226] Negotiated authn=JWT
0809 20:17:48.982464 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:48.983657 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:48.984993 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:48.985717 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:48.985853 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:48.986357 (+ 504us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 20:17:48.988761 (+ 2404us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46031/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 20:17:48.989069 (+ 308us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33520: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46031/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":73}
W20260809 20:17:48.989526 11726 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33520: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46031/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 20:17:48.991021 11759 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:17:48.980778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38327 (local address 127.0.0.1:33520)
0809 20:17:48.981440 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:48.981457 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:48.981579 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:48.982060 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:48.982063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:48.982072 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0809 20:17:48.982312 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:48.982319 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:48.983792 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:48.983796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:48.984846 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:48.984858 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:48.985853 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:48.985917 (+ 64us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:48.986052 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:48.986082 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 20:17:48.988963 (+ 2881us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46031/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 20:17:48.990609 (+ 1646us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.61.62:38327: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46031/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":1365,"thread_start_us":377,"threads_started":1}
I20260809 20:17:48.992007 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 11645
I20260809 20:17:48.993216 11714 generic_service.cc:196] Checking for leaks (request via RPC)
E20260809 20:17:49.001451 11760 webserver.cc:533] Webserver: sslize failed:
E20260809 20:17:49.001569 11760 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 20:17:49.083252 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 11645
2026-08-09T20:17:49Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (713 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T20:17:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:17:49Z Disabled control of system clock
I20260809 20:17:49.172950 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:42167
--webserver_interface=127.5.61.62
--webserver_port=0
--builtin_ntp_servers=127.5.61.20:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:42167
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786306623011809-5364-0/testchainca.cert with env {}
W20260809 20:17:49.502141 11765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:49.502418 11765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:49.502506 11765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:49.509313 11765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:17:49.509402 11765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:49.509439 11765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:49.509486 11765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:17:49.509522 11765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 20:17:49.509611 11765 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 20:17:49.509651 11765 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 20:17:49.518956 11765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:42167
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:42167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786306623011809-5364-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:49.520293 11765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:49.521447 11765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:49.533113 11773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:49.533182 11772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:49.533159 11775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:49.533592 11765 server_base.cc:1034] running on GCE node
I20260809 20:17:49.534535 11765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:49.535874 11765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:49.537106 11765 hybrid_clock.cc:648] HybridClock initialized: now 1786306669537054 us; error 34 us; skew 500 ppm
I20260809 20:17:49.537410 11765 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:17:49.547334 11765 webserver.cc:460] Webserver started at http://127.5.61.62:37931/ using document root <none> and password file <none>
I20260809 20:17:49.548089 11765 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:49.548185 11765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:49.548454 11765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:49.551270 11765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "2563cc14297e48618cc1210c5bf3b787"
format_stamp: "Formatted at 2026-08-09 20:17:49 on dist-test-slave-6k22"
server_key: "9f11f6ba0492f0d67d63a5052a110c7a"
server_key_iv: "8bce39c840b1766972b241b0bdc3a4bd"
server_key_version: "encryptionkey@0"
I20260809 20:17:49.552083 11765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "2563cc14297e48618cc1210c5bf3b787"
format_stamp: "Formatted at 2026-08-09 20:17:49 on dist-test-slave-6k22"
server_key: "9f11f6ba0492f0d67d63a5052a110c7a"
server_key_iv: "8bce39c840b1766972b241b0bdc3a4bd"
server_key_version: "encryptionkey@0"
I20260809 20:17:49.557772 11765 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 20:17:49.562024 11782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:49.563449 11765 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 20:17:49.563675 11765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "2563cc14297e48618cc1210c5bf3b787"
format_stamp: "Formatted at 2026-08-09 20:17:49 on dist-test-slave-6k22"
server_key: "9f11f6ba0492f0d67d63a5052a110c7a"
server_key_iv: "8bce39c840b1766972b241b0bdc3a4bd"
server_key_version: "encryptionkey@0"
I20260809 20:17:49.563897 11765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:49.608824 11765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:49.609722 11765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:49.610028 11765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:49.629537 11765 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:42167
I20260809 20:17:49.629558 11841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:42167 every 8 connection(s)
I20260809 20:17:49.630877 11765 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:17:49.634456 11842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:49.636543 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 11765
I20260809 20:17:49.636765 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:17:49.637607 5364 external_mini_cluster.cc:1428] Setting key b53bdc902eb8dafc57498f2f003b2650
I20260809 20:17:49.646665 11842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2563cc14297e48618cc1210c5bf3b787: Bootstrap starting.
I20260809 20:17:49.650187 11842 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2563cc14297e48618cc1210c5bf3b787: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:49.651065 11846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:49.639639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36386 (local address 127.5.61.62:42167)
0809 20:17:49.640687 (+ 1048us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:49.640696 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:49.640738 (+ 42us) server_negotiation.cc:407] Connection header received
0809 20:17:49.642327 (+ 1589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:49.642348 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:49.642773 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:49.643082 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:49.643676 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:49.644987 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:49.646812 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:49.647455 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:49.647651 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:49.648829 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:49.648836 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:49.648848 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:17:49.649367 (+ 519us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:49.649442 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:17:49.649461 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:49.649559 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:49.649783 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 20:17:49.650096 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":373,"threads_started":1}
I20260809 20:17:49.651268 11842 log.cc:824] T 00000000000000000000000000000000 P 2563cc14297e48618cc1210c5bf3b787: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:49.654488 11842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2563cc14297e48618cc1210c5bf3b787: No bootstrap required, opened a new log
I20260809 20:17:49.655290 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:17:49.662494 11842 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2563cc14297e48618cc1210c5bf3b787 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2563cc14297e48618cc1210c5bf3b787" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 42167 } }
I20260809 20:17:49.662823 11842 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2563cc14297e48618cc1210c5bf3b787 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:49.662952 11842 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2563cc14297e48618cc1210c5bf3b787 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2563cc14297e48618cc1210c5bf3b787, State: Initialized, Role: FOLLOWER
I20260809 20:17:49.663538 11842 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2563cc14297e48618cc1210c5bf3b787 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2563cc14297e48618cc1210c5bf3b787" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 42167 } }
I20260809 20:17:49.663798 11842 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2563cc14297e48618cc1210c5bf3b787 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:17:49.663901 11842 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2563cc14297e48618cc1210c5bf3b787 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:17:49.664021 11842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2563cc14297e48618cc1210c5bf3b787 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:49.665947 11842 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2563cc14297e48618cc1210c5bf3b787 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2563cc14297e48618cc1210c5bf3b787" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 42167 } }
I20260809 20:17:49.666358 11842 leader_election.cc:302] T 00000000000000000000000000000000 P 2563cc14297e48618cc1210c5bf3b787 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2563cc14297e48618cc1210c5bf3b787; no voters:
I20260809 20:17:49.666864 11842 leader_election.cc:288] T 00000000000000000000000000000000 P 2563cc14297e48618cc1210c5bf3b787 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:17:49.666990 11848 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2563cc14297e48618cc1210c5bf3b787 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:49.667837 11848 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2563cc14297e48618cc1210c5bf3b787 [term 1 LEADER]: Becoming Leader. State: Replica: 2563cc14297e48618cc1210c5bf3b787, State: Running, Role: LEADER
I20260809 20:17:49.668450 11848 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2563cc14297e48618cc1210c5bf3b787 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2563cc14297e48618cc1210c5bf3b787" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 42167 } }
I20260809 20:17:49.669052 11842 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2563cc14297e48618cc1210c5bf3b787 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:17:49.671507 11849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2563cc14297e48618cc1210c5bf3b787 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2563cc14297e48618cc1210c5bf3b787" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2563cc14297e48618cc1210c5bf3b787" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 42167 } } }
I20260809 20:17:49.671893 11849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2563cc14297e48618cc1210c5bf3b787 [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:49.671445 11850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2563cc14297e48618cc1210c5bf3b787 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2563cc14297e48618cc1210c5bf3b787. Latest consensus state: current_term: 1 leader_uuid: "2563cc14297e48618cc1210c5bf3b787" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2563cc14297e48618cc1210c5bf3b787" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 42167 } } }
I20260809 20:17:49.672727 11850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2563cc14297e48618cc1210c5bf3b787 [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:49.674350 11854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:17:49.680548 11854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:17:49.688663 11854 catalog_manager.cc:1347] Generated new cluster ID: 8120e1ca3ed14b289f2ae2f76f1f5ddb
I20260809 20:17:49.688766 11854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:17:49.714318 11854 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:17:49.716248 11854 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:17:49.730744 11854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2563cc14297e48618cc1210c5bf3b787: Generated new TSK 0
I20260809 20:17:49.731520 11854 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:17:50.223796 11846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:49.750330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36398 (local address 127.5.61.62:42167)
0809 20:17:49.750502 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:49.750509 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:49.751014 (+ 505us) server_negotiation.cc:407] Connection header received
0809 20:17:49.751230 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:49.751236 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:49.751325 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:49.751478 (+ 153us) server_negotiation.cc:226] Negotiated authn=JWT
0809 20:17:49.751976 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:49.753100 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:49.754406 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:49.755116 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:49.755241 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:49.755710 (+ 469us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 20:17:50.223014 (+467304us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 20:17:50.223461 (+ 447us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36398: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 20:17:50.223950 11846 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36398: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 20:17:50.224097 11875 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:17:49.750315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:42167 (local address 127.0.0.1:36398)
0809 20:17:49.750921 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:49.750936 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:49.751047 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:49.751496 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:49.751500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:49.751509 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0809 20:17:49.751830 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:49.751838 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:49.753228 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:49.753232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:49.754266 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:49.754277 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:49.755240 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:49.755282 (+ 42us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:49.755416 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:49.755446 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 20:17:50.223321 (+467875us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 20:17:50.223654 (+ 333us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.61.62:42167: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":341,"threads_started":1}
I20260809 20:17:50.225323 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 11765
I20260809 20:17:50.226442 11837 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:50.327633 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 11765
2026-08-09T20:17:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1225 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (410 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (403 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:17:51 dist-test-slave-6k22 krb5kdc[11894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:17:51 dist-test-slave-6k22 krb5kdc[11894](info): set up 2 sockets
Aug 09 20:17:51 dist-test-slave-6k22 krb5kdc[11894](info): commencing operation
krb5kdc: starting...
W20260809 20:17:52.313436 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:17:52 dist-test-slave-6k22 krb5kdc[11894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:17:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:17:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:17:52.539179 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:46031
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:36679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:46031
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:52.860867 11910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:52.861208 11910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:52.861369 11910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:52.868389 11910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:17:52.868520 11910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:52.868611 11910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:52.868748 11910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:17:52.868824 11910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:17:52.877907 11910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:36679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:46031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:46031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:52.879170 11910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:52.880533 11910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:52.891935 11919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:52.891903 11917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:52.891964 11916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:52.892654 11910 server_base.cc:1034] running on GCE node
I20260809 20:17:52.893435 11910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:52.894951 11910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:52.896183 11910 hybrid_clock.cc:648] HybridClock initialized: now 1786306672896135 us; error 55 us; skew 500 ppm
I20260809 20:17:52.896528 11910 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:52 dist-test-slave-6k22 krb5kdc[11894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306672, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:52.899775 11910 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:17:52.901050 11910 webserver.cc:460] Webserver started at http://127.5.61.62:39683/ using document root <none> and password file <none>
I20260809 20:17:52.901643 11910 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:52.901784 11910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:52.902052 11910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:52.904938 11910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "2777ff5d54c6444086c58887125350e9"
format_stamp: "Formatted at 2026-08-09 20:17:52 on dist-test-slave-6k22"
server_key: "18ef5ff17d34f0c166241b819810b354"
server_key_iv: "a92589f2af9e3a6506260af36a94775f"
server_key_version: "encryptionkey@0"
I20260809 20:17:52.905633 11910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "2777ff5d54c6444086c58887125350e9"
format_stamp: "Formatted at 2026-08-09 20:17:52 on dist-test-slave-6k22"
server_key: "18ef5ff17d34f0c166241b819810b354"
server_key_iv: "a92589f2af9e3a6506260af36a94775f"
server_key_version: "encryptionkey@0"
I20260809 20:17:52.910637 11910 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 20:17:52.913980 11927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:52.915234 11910 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 20:17:52.915401 11910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "2777ff5d54c6444086c58887125350e9"
format_stamp: "Formatted at 2026-08-09 20:17:52 on dist-test-slave-6k22"
server_key: "18ef5ff17d34f0c166241b819810b354"
server_key_iv: "a92589f2af9e3a6506260af36a94775f"
server_key_version: "encryptionkey@0"
I20260809 20:17:52.915597 11910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:52.961282 11910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:52.965760 11910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:52.966044 11910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:52.984707 11910 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:46031
I20260809 20:17:52.984751 11984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:46031 every 8 connection(s)
I20260809 20:17:52.986076 11910 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:17:52.989476 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 11910
I20260809 20:17:52.989717 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:17:52.989993 11985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:52.990434 5364 external_mini_cluster.cc:1428] Setting key 32c575db571edaeb4c0e31abb23a997e
I20260809 20:17:53.002281 11985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2777ff5d54c6444086c58887125350e9: Bootstrap starting.
Aug 09 20:17:53 dist-test-slave-6k22 krb5kdc[11894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:53.005365 11985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2777ff5d54c6444086c58887125350e9: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:53.006449 11985 log.cc:824] T 00000000000000000000000000000000 P 2777ff5d54c6444086c58887125350e9: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:53.009168 11985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2777ff5d54c6444086c58887125350e9: No bootstrap required, opened a new log
I20260809 20:17:53.012162 11988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:52.992420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44580 (local address 127.5.61.62:46031)
0809 20:17:52.993217 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:52.993228 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:52.993269 (+ 41us) server_negotiation.cc:407] Connection header received
0809 20:17:52.994933 (+ 1664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:52.994952 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:52.995317 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:52.995788 (+ 471us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:52.997234 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:52.998511 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:52.999844 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:53.000522 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:53.000734 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:53.004769 (+ 4035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:53.004776 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:53.004790 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:53.004828 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:53.008031 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:53.008702 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:53.008708 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:53.008713 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:53.008881 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:53.009281 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:53.009292 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:53.009294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:53.009793 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:53.010107 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:53.010548 (+ 441us) server_negotiation.cc:299] Negotiation successful
0809 20:17:53.010957 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":359,"threads_started":1}
I20260809 20:17:53.017791 11985 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2777ff5d54c6444086c58887125350e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2777ff5d54c6444086c58887125350e9" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46031 } }
I20260809 20:17:53.018136 11985 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2777ff5d54c6444086c58887125350e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:53.018242 11985 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2777ff5d54c6444086c58887125350e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2777ff5d54c6444086c58887125350e9, State: Initialized, Role: FOLLOWER
I20260809 20:17:53.018858 11985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2777ff5d54c6444086c58887125350e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2777ff5d54c6444086c58887125350e9" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46031 } }
I20260809 20:17:53.019105 11985 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2777ff5d54c6444086c58887125350e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:17:53.019253 11985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2777ff5d54c6444086c58887125350e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:17:53.019429 11985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2777ff5d54c6444086c58887125350e9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:53.021503 11985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2777ff5d54c6444086c58887125350e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2777ff5d54c6444086c58887125350e9" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46031 } }
I20260809 20:17:53.021925 11985 leader_election.cc:302] T 00000000000000000000000000000000 P 2777ff5d54c6444086c58887125350e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2777ff5d54c6444086c58887125350e9; no voters:
I20260809 20:17:53.022450 11985 leader_election.cc:288] T 00000000000000000000000000000000 P 2777ff5d54c6444086c58887125350e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:17:53.022550 11992 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2777ff5d54c6444086c58887125350e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:53.023276 11992 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2777ff5d54c6444086c58887125350e9 [term 1 LEADER]: Becoming Leader. State: Replica: 2777ff5d54c6444086c58887125350e9, State: Running, Role: LEADER
I20260809 20:17:53.023918 11992 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2777ff5d54c6444086c58887125350e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2777ff5d54c6444086c58887125350e9" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46031 } }
I20260809 20:17:53.024519 11985 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2777ff5d54c6444086c58887125350e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:17:53.026950 11993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2777ff5d54c6444086c58887125350e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2777ff5d54c6444086c58887125350e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2777ff5d54c6444086c58887125350e9" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46031 } } }
I20260809 20:17:53.027809 11993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2777ff5d54c6444086c58887125350e9 [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:53.027151 11994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2777ff5d54c6444086c58887125350e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2777ff5d54c6444086c58887125350e9. Latest consensus state: current_term: 1 leader_uuid: "2777ff5d54c6444086c58887125350e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2777ff5d54c6444086c58887125350e9" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46031 } } }
I20260809 20:17:53.027998 11994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2777ff5d54c6444086c58887125350e9 [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:53.029453 12001 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:17:53.034089 12001 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:17:53.042418 12001 catalog_manager.cc:1347] Generated new cluster ID: 8a7009a1d5df430cb1716ae8fa1b5e68
I20260809 20:17:53.042514 12001 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:17:53.084409 12001 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:17:53.086052 12001 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:17:53.097288 12001 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2777ff5d54c6444086c58887125350e9: Generated new TSK 0
I20260809 20:17:53.098104 12001 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.61.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
I20260809 20:17:53.152135 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:46031
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:36679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:53.483194 12015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:53.483455 12015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:53.483736 12015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:53.490919 12015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:53.491011 12015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:53.491150 12015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:17:53.501032 12015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:36679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:46031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:53.502254 12015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:53.503505 12015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:53.516289 12022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:53.516392 12024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:53.516378 12021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:53.520989 12015 server_base.cc:1034] running on GCE node
I20260809 20:17:53.521588 12015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:53.522465 12015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:53.523650 12015 hybrid_clock.cc:648] HybridClock initialized: now 1786306673523594 us; error 35 us; skew 500 ppm
I20260809 20:17:53.523962 12015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:53 dist-test-slave-6k22 krb5kdc[11894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306673, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:53.527345 12015 init.cc:377] Logged in from keytab as kudu/127.5.61.1@KRBTEST.COM (short username kudu)
I20260809 20:17:53.529008 12015 webserver.cc:460] Webserver started at http://127.5.61.1:46387/ using document root <none> and password file <none>
I20260809 20:17:53.529731 12015 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:53.529863 12015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:53.530197 12015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:53.533164 12015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "571f33c1c2874ffa8d7a815195112c8e"
format_stamp: "Formatted at 2026-08-09 20:17:53 on dist-test-slave-6k22"
server_key: "51323f3a7ea5f5ec955dd2483d6d855e"
server_key_iv: "d71a57747716a6ba355763c2e29cc511"
server_key_version: "encryptionkey@0"
I20260809 20:17:53.533931 12015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "571f33c1c2874ffa8d7a815195112c8e"
format_stamp: "Formatted at 2026-08-09 20:17:53 on dist-test-slave-6k22"
server_key: "51323f3a7ea5f5ec955dd2483d6d855e"
server_key_iv: "d71a57747716a6ba355763c2e29cc511"
server_key_version: "encryptionkey@0"
I20260809 20:17:53.538853 12015 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260809 20:17:53.542192 12033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:53.543627 12015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 20:17:53.543761 12015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "571f33c1c2874ffa8d7a815195112c8e"
format_stamp: "Formatted at 2026-08-09 20:17:53 on dist-test-slave-6k22"
server_key: "51323f3a7ea5f5ec955dd2483d6d855e"
server_key_iv: "d71a57747716a6ba355763c2e29cc511"
server_key_version: "encryptionkey@0"
I20260809 20:17:53.543926 12015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:53.557823 12015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:53.561545 12015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:53.561779 12015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:53.562650 12015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:53.563948 12015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:53.564023 12015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:53.564109 12015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:53.564157 12015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:53.594707 12015 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:44927
I20260809 20:17:53.594751 12167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:44927 every 8 connection(s)
I20260809 20:17:53.596014 12015 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:17:53.602800 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 12015
I20260809 20:17:53.603000 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:17:53.603853 5364 external_mini_cluster.cc:1428] Setting key 7b181510548fdfc6bf77f8621747af74
Aug 09 20:17:53 dist-test-slave-6k22 krb5kdc[11894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306673, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:53.618284 12171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:53.598780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:32945 (local address 127.5.61.62:46031)
0809 20:17:53.599232 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:53.599239 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:53.600260 (+ 1021us) server_negotiation.cc:407] Connection header received
0809 20:17:53.602364 (+ 2104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:53.602371 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:53.602457 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:53.602576 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:53.604984 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:53.605905 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:53.607617 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:53.608052 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:53.608186 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:53.611892 (+ 3706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:53.611901 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:53.611909 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:53.611957 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:53.614957 (+ 3000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:53.615916 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:53.615920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:53.615923 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:53.616055 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:53.616557 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:53.616560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:53.616562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:53.616818 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:53.616928 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:53.617820 (+ 892us) server_negotiation.cc:299] Negotiation successful
0809 20:17:53.617976 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":208,"threads_started":1}
I20260809 20:17:53.619910 12172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:53.599306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46031 (local address 127.5.61.1:32945)
0809 20:17:53.600093 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:53.600152 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:53.602105 (+ 1953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:53.602850 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:53.602858 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:53.603787 (+ 929us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:53.604813 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:53.604834 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:53.606141 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:53.606145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:53.607459 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:53.607471 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:53.608289 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:53.608298 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:53.608536 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:53.609209 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:53.609231 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:53.611673 (+ 2442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:53.615095 (+ 3422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:53.615106 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:53.615125 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:53.615802 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:53.616181 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:53.616189 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:53.616196 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:53.616448 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:53.616943 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:53.616953 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:53.617406 (+ 453us) client_negotiation.cc:769] Sending connection context
0809 20:17:53.617748 (+ 342us) client_negotiation.cc:240] Negotiation successful
0809 20:17:53.618321 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":369,"threads_started":1}
I20260809 20:17:53.621568 12168 heartbeater.cc:347] Connected to a master server at 127.5.61.62:46031
I20260809 20:17:53.621878 12168 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:53.622654 12168 heartbeater.cc:511] Master 127.5.61.62:46031 requested a full tablet report, sending...
I20260809 20:17:53.624792 11947 ts_manager.cc:194] Registered new tserver with Master: 571f33c1c2874ffa8d7a815195112c8e (127.5.61.1:44927)
I20260809 20:17:53.626283 11947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.1@KRBTEST.COM'} at 127.5.61.1:32945
WARNING: no policy specified for kudu/127.5.61.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
I20260809 20:17:53.659155 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:46031
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:36679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:54.003780 12176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:54.004161 12176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:54.004307 12176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:54.012737 12176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:54.012851 12176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:54.012967 12176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:17:54.022838 12176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:36679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:46031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:54.024139 12176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:54.025511 12176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:54.038774 12181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:54.038743 12182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:54.038708 12184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:54.039599 12176 server_base.cc:1034] running on GCE node
I20260809 20:17:54.040206 12176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:54.041142 12176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:54.042354 12176 hybrid_clock.cc:648] HybridClock initialized: now 1786306674042301 us; error 31 us; skew 500 ppm
I20260809 20:17:54.042675 12176 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:54 dist-test-slave-6k22 krb5kdc[11894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306674, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:54.045743 12176 init.cc:377] Logged in from keytab as kudu/127.5.61.2@KRBTEST.COM (short username kudu)
I20260809 20:17:54.047183 12176 webserver.cc:460] Webserver started at http://127.5.61.2:40557/ using document root <none> and password file <none>
I20260809 20:17:54.047930 12176 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:54.048038 12176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:54.048286 12176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:54.051074 12176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "80e3702d41554d0b9874a44dbc92490e"
format_stamp: "Formatted at 2026-08-09 20:17:54 on dist-test-slave-6k22"
server_key: "67f6d3cbbc0f27f1777e9e1c19589246"
server_key_iv: "9e2df568c10f66c6d599487db9e00c27"
server_key_version: "encryptionkey@0"
I20260809 20:17:54.051873 12176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "80e3702d41554d0b9874a44dbc92490e"
format_stamp: "Formatted at 2026-08-09 20:17:54 on dist-test-slave-6k22"
server_key: "67f6d3cbbc0f27f1777e9e1c19589246"
server_key_iv: "9e2df568c10f66c6d599487db9e00c27"
server_key_version: "encryptionkey@0"
I20260809 20:17:54.057112 12176 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 20:17:54.060459 12191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:54.061795 12176 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:17:54.062000 12176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "80e3702d41554d0b9874a44dbc92490e"
format_stamp: "Formatted at 2026-08-09 20:17:54 on dist-test-slave-6k22"
server_key: "67f6d3cbbc0f27f1777e9e1c19589246"
server_key_iv: "9e2df568c10f66c6d599487db9e00c27"
server_key_version: "encryptionkey@0"
I20260809 20:17:54.062213 12176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:54.075178 12176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:54.078982 12176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:54.079279 12176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:54.080283 12176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:54.081585 12176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:54.081693 12176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:54.081840 12176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:54.081920 12176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:54.117215 12176 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:35385
I20260809 20:17:54.117249 12317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:35385 every 8 connection(s)
I20260809 20:17:54.118732 12176 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:17:54.120785 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 12176
I20260809 20:17:54.120965 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:17:54.121444 5364 external_mini_cluster.cc:1428] Setting key 4ddcf9e196250ddb5d54b4363372b86c
Aug 09 20:17:54 dist-test-slave-6k22 krb5kdc[11894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306674, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:54.141592 12320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:54.121765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:55431 (local address 127.5.61.62:46031)
0809 20:17:54.122465 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:54.122470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:54.123691 (+ 1221us) server_negotiation.cc:407] Connection header received
0809 20:17:54.125714 (+ 2023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:54.125719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:54.125808 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:54.125932 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:54.127441 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:54.128246 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:54.131321 (+ 3075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:54.131888 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:54.132027 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:54.135659 (+ 3632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:54.135664 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:54.135668 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:54.135700 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:54.138391 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:54.139121 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:54.139128 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:54.139131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:54.139275 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:54.139785 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:54.139791 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:54.139795 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:54.140052 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:54.140208 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:54.140974 (+ 766us) server_negotiation.cc:299] Negotiation successful
0809 20:17:54.141194 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":468,"threads_started":1}
I20260809 20:17:54.142597 12321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:54.122473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46031 (local address 127.5.61.2:55431)
0809 20:17:54.123470 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:54.123530 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:54.125342 (+ 1812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:54.126171 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:54.126179 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:54.126690 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:54.127258 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:54.127276 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:54.128423 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:54.128433 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:54.129850 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:54.131146 (+ 1296us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:54.132145 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:54.132151 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:54.132364 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:54.133120 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:54.133145 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:54.135390 (+ 2245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:54.138568 (+ 3178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:54.138576 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:54.138590 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:54.138991 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:54.139400 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:54.139405 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:54.139409 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:54.139671 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:54.140205 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:54.140212 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:54.140565 (+ 353us) client_negotiation.cc:769] Sending connection context
0809 20:17:54.140891 (+ 326us) client_negotiation.cc:240] Negotiation successful
0809 20:17:54.141424 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":373,"threads_started":1}
I20260809 20:17:54.144264 12318 heartbeater.cc:347] Connected to a master server at 127.5.61.62:46031
I20260809 20:17:54.144575 12318 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.5.61.3@KRBTEST.COM; defaulting to no policy
I20260809 20:17:54.145359 12318 heartbeater.cc:511] Master 127.5.61.62:46031 requested a full tablet report, sending...
I20260809 20:17:54.146905 11947 ts_manager.cc:194] Registered new tserver with Master: 80e3702d41554d0b9874a44dbc92490e (127.5.61.2:35385)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.3@KRBTEST.COM" created.
I20260809 20:17:54.147893 11947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.2@KRBTEST.COM'} at 127.5.61.2:55431
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
I20260809 20:17:54.176090 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:46031
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:36679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:54.517145 12324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:54.517408 12324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:54.517567 12324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:54.524890 12324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:54.525005 12324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:54.525182 12324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:17:54.534806 12324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:36679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:46031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:54.536125 12324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:54.537451 12324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:54.550477 12329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:54.550577 12333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:54.550619 12330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:54.551644 12324 server_base.cc:1034] running on GCE node
I20260809 20:17:54.552258 12324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:54.553313 12324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:54.554595 12324 hybrid_clock.cc:648] HybridClock initialized: now 1786306674554556 us; error 68 us; skew 500 ppm
I20260809 20:17:54.554967 12324 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:54 dist-test-slave-6k22 krb5kdc[11894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306674, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:54.562750 12324 init.cc:377] Logged in from keytab as kudu/127.5.61.3@KRBTEST.COM (short username kudu)
I20260809 20:17:54.564410 12324 webserver.cc:460] Webserver started at http://127.5.61.3:35879/ using document root <none> and password file <none>
I20260809 20:17:54.565299 12324 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:54.565507 12324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:54.565922 12324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:54.569185 12324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "348f5e6dd18b42ddb9ec3e0c41e42686"
format_stamp: "Formatted at 2026-08-09 20:17:54 on dist-test-slave-6k22"
server_key: "01c96c652251a972525733bc0b1bc7c7"
server_key_iv: "67e80a9254678dbbcd9b71db9e92e375"
server_key_version: "encryptionkey@0"
I20260809 20:17:54.569970 12324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "348f5e6dd18b42ddb9ec3e0c41e42686"
format_stamp: "Formatted at 2026-08-09 20:17:54 on dist-test-slave-6k22"
server_key: "01c96c652251a972525733bc0b1bc7c7"
server_key_iv: "67e80a9254678dbbcd9b71db9e92e375"
server_key_version: "encryptionkey@0"
I20260809 20:17:54.574965 12324 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 20:17:54.578281 12341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:54.579705 12324 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:17:54.579874 12324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "348f5e6dd18b42ddb9ec3e0c41e42686"
format_stamp: "Formatted at 2026-08-09 20:17:54 on dist-test-slave-6k22"
server_key: "01c96c652251a972525733bc0b1bc7c7"
server_key_iv: "67e80a9254678dbbcd9b71db9e92e375"
server_key_version: "encryptionkey@0"
I20260809 20:17:54.580106 12324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:54.603725 12324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:54.607470 12324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:54.607781 12324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:54.608734 12324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:54.610055 12324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:54.610160 12324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:54.610327 12324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:54.610400 12324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:54.629379 12168 heartbeater.cc:503] Master 127.5.61.62:46031 was elected leader, sending a full tablet report...
I20260809 20:17:54.645170 12324 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:38897
I20260809 20:17:54.645256 12473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:38897 every 8 connection(s)
I20260809 20:17:54.646701 12324 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:17:54.653009 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 12324
I20260809 20:17:54.653218 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:17:54.653664 5364 external_mini_cluster.cc:1428] Setting key 2be3464f087b8358787d19962131eded
Aug 09 20:17:54 dist-test-slave-6k22 krb5kdc[11894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306674, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:17:54.666177 12476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:54.649356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:39567 (local address 127.5.61.62:46031)
0809 20:17:54.649888 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:54.649892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:54.650689 (+ 797us) server_negotiation.cc:407] Connection header received
0809 20:17:54.652761 (+ 2072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:54.652767 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:54.652873 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:54.652998 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:54.654683 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:54.655672 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:54.657347 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:54.657745 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:54.657839 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:54.661089 (+ 3250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:54.661094 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:54.661097 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:54.661127 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:54.663553 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:54.664252 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:54.664256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:54.664258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:54.664372 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:54.664723 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:54.664726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:54.664727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:54.664966 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:54.665078 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:54.665652 (+ 574us) server_negotiation.cc:299] Negotiation successful
0809 20:17:54.665864 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":276,"threads_started":1}
I20260809 20:17:54.667043 12477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:54.649792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46031 (local address 127.5.61.3:39567)
0809 20:17:54.650510 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:54.650559 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:54.652442 (+ 1883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:54.653242 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:54.653251 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:54.653864 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:54.654488 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:54.654512 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:54.655845 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:54.655854 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:54.657220 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:54.657232 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:54.657927 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:54.657931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:54.658134 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:54.658735 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:54.658756 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:54.660909 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:54.663735 (+ 2826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:54.663741 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:54.663751 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:54.664154 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:54.664462 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:54.664465 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:54.664467 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:54.664632 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:54.665075 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:54.665079 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:54.665338 (+ 259us) client_negotiation.cc:769] Sending connection context
0809 20:17:54.665566 (+ 228us) client_negotiation.cc:240] Negotiation successful
0809 20:17:54.665914 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":291,"threads_started":1}
I20260809 20:17:54.668390 12474 heartbeater.cc:347] Connected to a master server at 127.5.61.62:46031
I20260809 20:17:54.668685 12474 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:54.669463 12474 heartbeater.cc:511] Master 127.5.61.62:46031 requested a full tablet report, sending...
I20260809 20:17:54.670881 11945 ts_manager.cc:194] Registered new tserver with Master: 348f5e6dd18b42ddb9ec3e0c41e42686 (127.5.61.3:38897)
I20260809 20:17:54.671850 11945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.3@KRBTEST.COM'} at 127.5.61.3:39567
I20260809 20:17:54.681242 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/test-admin.keytab.
Aug 09 20:17:54 dist-test-slave-6k22 krb5kdc[11894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:54.713644 5364 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 20:17:54.718992 12482 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:17:54.716393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54198 (local address 127.5.61.1:44927)
0809 20:17:54.717046 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:54.717053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:54.717081 (+ 28us) server_negotiation.cc:407] Connection header received
0809 20:17:54.717140 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:54.717147 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:54.717373 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:54.717501 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:54.718510 (+ 1009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54198: BlockingRecv error: recv got EOF from 127.0.0.1:54198 (error 108)
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":345,"threads_started":1}
W20260809 20:17:54.718997 5364 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/krb5cc' of type FILE
Aug 09 20:17:54 dist-test-slave-6k22 krb5kdc[11894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:54.720638 5364 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 20:17:54 dist-test-slave-6k22 krb5kdc[11894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:54.722018 5364 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 20:17:54 dist-test-slave-6k22 krb5kdc[11894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.1@KRBTEST.COM
I20260809 20:17:54.736934 12482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:54.723642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54212 (local address 127.5.61.1:44927)
0809 20:17:54.723816 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:54.723839 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:54.724006 (+ 167us) server_negotiation.cc:407] Connection header received
0809 20:17:54.724123 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:54.724127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:54.724179 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:54.724256 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:54.725308 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:54.726315 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:54.727513 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:54.728095 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:54.728189 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:54.731919 (+ 3730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:54.731930 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:54.731942 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:54.731972 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:54.734555 (+ 2583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:54.735163 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:54.735170 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:54.735172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:54.735316 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:54.735677 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:54.735680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:54.735682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:54.736004 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:54.736227 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:54.736496 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 20:17:54.736680 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 20:17:54.742527 12482 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:17:54.740724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54216 (local address 127.5.61.1:44927)
0809 20:17:54.740947 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:54.740953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:54.741252 (+ 299us) server_negotiation.cc:407] Connection header received
0809 20:17:54.741463 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:54.741468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:54.741545 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:54.741644 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:54.742307 (+ 663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54216: BlockingRecv error: recv got EOF from 127.0.0.1:54216 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260809 20:17:54.742722 5364 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/krb5cc' of type FILE
Aug 09 20:17:54 dist-test-slave-6k22 krb5kdc[11894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:54.744122 5364 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 20:17:54 dist-test-slave-6k22 krb5kdc[11894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:54.745407 5364 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 20:17:54 dist-test-slave-6k22 krb5kdc[11894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.1@KRBTEST.COM
I20260809 20:17:54.758462 12482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:54.746571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54222 (local address 127.5.61.1:44927)
0809 20:17:54.746751 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:54.746756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:54.747157 (+ 401us) server_negotiation.cc:407] Connection header received
0809 20:17:54.747261 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:54.747264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:54.747311 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:54.747397 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:54.748419 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:54.749115 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:54.750251 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:54.750652 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:54.750756 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:54.754035 (+ 3279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:54.754039 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:54.754042 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:54.754082 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:54.756438 (+ 2356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:54.757023 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:54.757027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:54.757029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:54.757138 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:54.757444 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:54.757447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:54.757448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:54.757686 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:54.757871 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:54.758096 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 20:17:54.758246 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260809 20:17:54.762465 12482 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:17:54.760836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54228 (local address 127.5.61.1:44927)
0809 20:17:54.761023 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:54.761029 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:54.761201 (+ 172us) server_negotiation.cc:407] Connection header received
0809 20:17:54.761374 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:54.761379 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:54.761450 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:54.761545 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:54.762248 (+ 703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54228: BlockingRecv error: recv got EOF from 127.0.0.1:54228 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260809 20:17:54.762722 5364 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786306623011809-5364-0/krb5kdc/krb5cc' of type FILE
Aug 09 20:17:54 dist-test-slave-6k22 krb5kdc[11894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:54.764158 5364 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 20:17:54 dist-test-slave-6k22 krb5kdc[11894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:54.765353 5364 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 20:17:54 dist-test-slave-6k22 krb5kdc[11894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.1@KRBTEST.COM
I20260809 20:17:54.777900 12482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:54.766505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54238 (local address 127.5.61.1:44927)
0809 20:17:54.766717 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:54.766722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:54.766970 (+ 248us) server_negotiation.cc:407] Connection header received
0809 20:17:54.767082 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:54.767085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:54.767132 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:54.767207 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:54.768103 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:54.768745 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:54.769838 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:54.770204 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:54.770290 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:54.773404 (+ 3114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:54.773408 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:54.773411 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:54.773437 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:54.775772 (+ 2335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:54.776334 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:54.776338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:54.776340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:54.776480 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:54.776785 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:54.776789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:54.776790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:54.777019 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:54.777218 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:54.777432 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 20:17:54.777624 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 20:17:54.779403 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 12015
I20260809 20:17:54.790723 12482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:54.780144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54240 (local address 127.5.61.1:44927)
0809 20:17:54.780357 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:54.780363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:54.780611 (+ 248us) server_negotiation.cc:407] Connection header received
0809 20:17:54.780773 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:54.780778 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:54.780834 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:54.780934 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:54.781809 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:54.782441 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:54.783727 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:54.784110 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:54.784234 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:54.786132 (+ 1898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:54.786138 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:54.786142 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:54.786178 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:54.788288 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:54.788901 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:54.788906 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:54.788909 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:54.789031 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:54.789410 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:54.789416 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:54.789419 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:54.789646 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:54.789906 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:54.790232 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 20:17:54.790440 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 20:17:54.791285 12163 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:54.874483 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 12015
I20260809 20:17:54.895128 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 12176
Aug 09 20:17:54 dist-test-slave-6k22 krb5kdc[11894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.2@KRBTEST.COM
I20260809 20:17:54.911128 12496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:54.896151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42708 (local address 127.5.61.2:35385)
0809 20:17:54.896748 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:54.896755 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:54.896780 (+ 25us) server_negotiation.cc:407] Connection header received
0809 20:17:54.896843 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:54.896850 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:54.897062 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:54.897193 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:54.898397 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:54.899511 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:54.900949 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:54.901555 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:54.901700 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:54.905535 (+ 3835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:54.905547 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:54.905561 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:54.905593 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:54.908374 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:54.909016 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:54.909028 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:54.909030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:54.909145 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:54.909546 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:54.909552 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:54.909555 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:54.909886 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:54.910209 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:54.910523 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 20:17:54.910806 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":320,"threads_started":1}
I20260809 20:17:54.912465 12313 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:55.005757 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 12176
I20260809 20:17:55.026522 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 12324
Aug 09 20:17:55 dist-test-slave-6k22 krb5kdc[11894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.3@KRBTEST.COM
I20260809 20:17:55.041970 12498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:55.027390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41822 (local address 127.5.61.3:38897)
0809 20:17:55.028053 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:55.028060 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:55.028112 (+ 52us) server_negotiation.cc:407] Connection header received
0809 20:17:55.028191 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:55.028199 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:55.028461 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:55.028623 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:55.029706 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:55.030846 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:55.032196 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:55.032726 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:55.032824 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:55.036657 (+ 3833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:55.036668 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:55.036682 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:55.036718 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:55.039373 (+ 2655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:55.040176 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:55.040183 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:55.040185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:55.040301 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:55.040620 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:55.040623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:55.040625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:55.040954 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:55.041184 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:55.041368 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 20:17:55.041651 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":348,"threads_started":1}
I20260809 20:17:55.043222 12468 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:55.124616 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 12324
I20260809 20:17:55.143535 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 11910
I20260809 20:17:55.144882 11979 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:17:55.246037 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 11910
2026-08-09T20:17:55Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4108 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:17:55 dist-test-slave-6k22 krb5kdc[12503](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:17:55 dist-test-slave-6k22 krb5kdc[12503](info): set up 2 sockets
Aug 09 20:17:55 dist-test-slave-6k22 krb5kdc[12503](info): commencing operation
krb5kdc: starting...
W20260809 20:17:56.412750 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.007s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:17:56 dist-test-slave-6k22 krb5kdc[12503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:17:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:17:56Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.62.keytab.
Entry for principal oryx/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.62.keytab.
I20260809 20:17:56.638991 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:40647
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.62.keytab
--principal=oryx/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:40647
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:56.962838 12521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:56.963179 12521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:56.963354 12521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:56.970543 12521 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:17:56.970697 12521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:56.970811 12521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:56.970914 12521 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:17:56.970997 12521 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:17:56.980616 12521 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:40647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.62.keytab
--principal=oryx/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:40647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:56.981978 12521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:56.983315 12521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:56.995340 12530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:56.995404 12528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:56.995402 12527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:56.996309 12521 server_base.cc:1034] running on GCE node
I20260809 20:17:56.997076 12521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:56.998412 12521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:56.999647 12521 hybrid_clock.cc:648] HybridClock initialized: now 1786306676999590 us; error 50 us; skew 500 ppm
I20260809 20:17:57.000001 12521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:56 dist-test-slave-6k22 krb5kdc[12503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306676, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:57.003157 12521 init.cc:377] Logged in from keytab as oryx/127.5.61.62@KRBTEST.COM (short username oryx)
I20260809 20:17:57.004575 12521 webserver.cc:460] Webserver started at http://127.5.61.62:38077/ using document root <none> and password file <none>
I20260809 20:17:57.005277 12521 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:57.005424 12521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:57.005751 12521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:57.008817 12521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "6798f13fda9446ddae84977ff493a57e"
format_stamp: "Formatted at 2026-08-09 20:17:57 on dist-test-slave-6k22"
server_key: "07f90a9ea543fd75f75819c9808c4c48"
server_key_iv: "81baa71c557e21622051d2fed47f4d58"
server_key_version: "encryptionkey@0"
I20260809 20:17:57.009600 12521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "6798f13fda9446ddae84977ff493a57e"
format_stamp: "Formatted at 2026-08-09 20:17:57 on dist-test-slave-6k22"
server_key: "07f90a9ea543fd75f75819c9808c4c48"
server_key_iv: "81baa71c557e21622051d2fed47f4d58"
server_key_version: "encryptionkey@0"
I20260809 20:17:57.014796 12521 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 20:17:57.018214 12541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:57.019536 12521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 20:17:57.019771 12521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "6798f13fda9446ddae84977ff493a57e"
format_stamp: "Formatted at 2026-08-09 20:17:57 on dist-test-slave-6k22"
server_key: "07f90a9ea543fd75f75819c9808c4c48"
server_key_iv: "81baa71c557e21622051d2fed47f4d58"
server_key_version: "encryptionkey@0"
I20260809 20:17:57.019969 12521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:57.040398 12521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:57.044472 12521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:57.044818 12521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:57.064793 12521 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:40647
I20260809 20:17:57.064828 12599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:40647 every 8 connection(s)
I20260809 20:17:57.066210 12521 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:17:57.069895 12600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:57.072757 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 12521
I20260809 20:17:57.072984 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:17:57.073624 5364 external_mini_cluster.cc:1428] Setting key 2dd320b48f69d75fdd7233e3aaa66662
I20260809 20:17:57.081998 12600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e: Bootstrap starting.
I20260809 20:17:57.085799 12600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:57.087149 12600 log.cc:824] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e: Log is configured to *not* fsync() on all Append() calls
Aug 09 20:17:57 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM
I20260809 20:17:57.090171 12600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e: No bootstrap required, opened a new log
I20260809 20:17:57.096427 12603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:57.075604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40038 (local address 127.5.61.62:40647)
0809 20:17:57.076351 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:57.076367 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:57.076407 (+ 40us) server_negotiation.cc:407] Connection header received
0809 20:17:57.077977 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:57.077997 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:57.078408 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:57.078731 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:57.080511 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:57.081793 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:57.083513 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:57.084231 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:57.084414 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:57.089071 (+ 4657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:57.089079 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:57.089092 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:57.089131 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:57.092299 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:57.092961 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:57.092974 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:57.092982 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:57.093145 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:57.093570 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:57.093574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:57.093578 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:57.094051 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:57.094433 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:57.094885 (+ 452us) server_negotiation.cc:299] Negotiation successful
0809 20:17:57.095304 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":344,"threads_started":1}
I20260809 20:17:57.098124 12600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } }
I20260809 20:17:57.098516 12600 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:57.098625 12600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6798f13fda9446ddae84977ff493a57e, State: Initialized, Role: FOLLOWER
I20260809 20:17:57.099342 12600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } }
I20260809 20:17:57.099640 12600 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:17:57.099786 12600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:17:57.099963 12600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:57.102403 12600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } }
I20260809 20:17:57.102909 12600 leader_election.cc:302] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6798f13fda9446ddae84977ff493a57e; no voters:
I20260809 20:17:57.103638 12600 leader_election.cc:288] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:17:57.103771 12605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:17:57.104499 12605 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [term 1 LEADER]: Becoming Leader. State: Replica: 6798f13fda9446ddae84977ff493a57e, State: Running, Role: LEADER
I20260809 20:17:57.105082 12605 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } }
I20260809 20:17:57.105957 12600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:17:57.107960 12606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6798f13fda9446ddae84977ff493a57e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } }
I20260809 20:17:57.108026 12607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6798f13fda9446ddae84977ff493a57e. Latest consensus state: current_term: 1 leader_uuid: "6798f13fda9446ddae84977ff493a57e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } }
I20260809 20:17:57.108374 12606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:57.108517 12607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:57.112900 12612 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:17:57.117184 12612 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:17:57.125336 12612 catalog_manager.cc:1347] Generated new cluster ID: 19ac0cc23316463f8eba2700c4b6ac78
I20260809 20:17:57.125450 12612 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:17:57.154451 12612 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:17:57.155846 12612 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:17:57.163821 12612 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e: Generated new TSK 0
I20260809 20:17:57.164635 12612 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.61.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.61.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.61.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.1.keytab.
Entry for principal oryx/127.5.61.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.1.keytab.
I20260809 20:17:57.226778 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:40647
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.1.keytab
--principal=oryx/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:57.554208 12626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:57.554471 12626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:57.554559 12626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:57.561486 12626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:57.561581 12626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:57.561697 12626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:17:57.570887 12626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.1.keytab
--principal=oryx/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:40647
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:57.572068 12626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:57.573310 12626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:57.585872 12631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:57.585901 12634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:57.585994 12632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:57.586652 12626 server_base.cc:1034] running on GCE node
I20260809 20:17:57.587232 12626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:57.588205 12626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:57.589392 12626 hybrid_clock.cc:648] HybridClock initialized: now 1786306677589338 us; error 51 us; skew 500 ppm
I20260809 20:17:57.589709 12626 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:57 dist-test-slave-6k22 krb5kdc[12503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306677, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:57.593022 12626 init.cc:377] Logged in from keytab as oryx/127.5.61.1@KRBTEST.COM (short username oryx)
I20260809 20:17:57.594290 12626 webserver.cc:460] Webserver started at http://127.5.61.1:39695/ using document root <none> and password file <none>
I20260809 20:17:57.594909 12626 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:57.594998 12626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:57.595271 12626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:57.598348 12626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "5b8af4b6cf364736a928ce119438e9a3"
format_stamp: "Formatted at 2026-08-09 20:17:57 on dist-test-slave-6k22"
server_key: "2f09bb317b7c221e65440c8c0b08f1e4"
server_key_iv: "ba21d555a17410a81b4a0b079fca3238"
server_key_version: "encryptionkey@0"
I20260809 20:17:57.599115 12626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "5b8af4b6cf364736a928ce119438e9a3"
format_stamp: "Formatted at 2026-08-09 20:17:57 on dist-test-slave-6k22"
server_key: "2f09bb317b7c221e65440c8c0b08f1e4"
server_key_iv: "ba21d555a17410a81b4a0b079fca3238"
server_key_version: "encryptionkey@0"
I20260809 20:17:57.604142 12626 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 20:17:57.608124 12641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:57.609408 12626 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 20:17:57.609575 12626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "5b8af4b6cf364736a928ce119438e9a3"
format_stamp: "Formatted at 2026-08-09 20:17:57 on dist-test-slave-6k22"
server_key: "2f09bb317b7c221e65440c8c0b08f1e4"
server_key_iv: "ba21d555a17410a81b4a0b079fca3238"
server_key_version: "encryptionkey@0"
I20260809 20:17:57.609787 12626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:57.638201 12626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:57.642566 12626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:57.642864 12626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:57.643796 12626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:57.645103 12626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:57.645215 12626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:57.645362 12626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:57.645445 12626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:57.679764 12626 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:37061
I20260809 20:17:57.679807 12778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:37061 every 8 connection(s)
I20260809 20:17:57.681017 12626 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:17:57.688231 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 12626
I20260809 20:17:57.688444 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:17:57.689113 5364 external_mini_cluster.cc:1428] Setting key 0523911b515608344f6e26a62122dbce
Aug 09 20:17:57 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306677, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM
I20260809 20:17:57.700827 12781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:57.683652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:57663 (local address 127.5.61.62:40647)
0809 20:17:57.684138 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:57.684142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:57.684927 (+ 785us) server_negotiation.cc:407] Connection header received
0809 20:17:57.686706 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:57.686711 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:57.686801 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:57.686928 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:57.688515 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:57.689322 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:57.691148 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:57.691599 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:57.691761 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:57.695176 (+ 3415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:57.695182 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:57.695186 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:57.695225 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:57.697801 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:57.698523 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:57.698529 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:57.698533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:57.698659 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:57.699131 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:57.699137 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:57.699140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:57.699395 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:57.699600 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:57.700248 (+ 648us) server_negotiation.cc:299] Negotiation successful
0809 20:17:57.700466 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":273,"threads_started":1}
I20260809 20:17:57.701746 12782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:57.684076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.5.61.1:57663)
0809 20:17:57.684763 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:57.684811 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:57.686430 (+ 1619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:57.687169 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:57.687178 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:57.687742 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:57.688296 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:57.688348 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:57.689490 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:57.689496 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:57.690997 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:57.691009 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:57.691841 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:57.691851 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:57.692085 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:57.692789 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:57.692810 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:57.694962 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:57.697937 (+ 2975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:57.697945 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:57.697959 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:57.698386 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:57.698814 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:57.698819 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:57.698822 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:57.699013 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:57.699539 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:57.699546 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:57.699896 (+ 350us) client_negotiation.cc:769] Sending connection context
0809 20:17:57.700149 (+ 253us) client_negotiation.cc:240] Negotiation successful
0809 20:17:57.700640 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":293,"threads_started":1}
I20260809 20:17:57.703306 12779 heartbeater.cc:347] Connected to a master server at 127.5.61.62:40647
I20260809 20:17:57.703678 12779 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:57.704567 12779 heartbeater.cc:511] Master 127.5.61.62:40647 requested a full tablet report, sending...
I20260809 20:17:57.707124 12559 ts_manager.cc:194] Registered new tserver with Master: 5b8af4b6cf364736a928ce119438e9a3 (127.5.61.1:37061)
I20260809 20:17:57.709388 12559 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.61.1@KRBTEST.COM'} at 127.5.61.1:57663
WARNING: no policy specified for oryx/127.5.61.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.61.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.61.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.2.keytab.
Entry for principal oryx/127.5.61.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.2.keytab.
I20260809 20:17:57.743961 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:40647
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.2.keytab
--principal=oryx/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:58.080901 12785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:58.081254 12785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:58.081422 12785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:58.090793 12785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:58.090895 12785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:58.091015 12785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:17:58.100493 12785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.2.keytab
--principal=oryx/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:40647
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:58.101665 12785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:58.102893 12785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:58.116662 12790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:58.116597 12791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:58.116736 12793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:58.117774 12785 server_base.cc:1034] running on GCE node
I20260809 20:17:58.118395 12785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:58.119310 12785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:58.120553 12785 hybrid_clock.cc:648] HybridClock initialized: now 1786306678120506 us; error 34 us; skew 500 ppm
I20260809 20:17:58.120945 12785 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:58 dist-test-slave-6k22 krb5kdc[12503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306678, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:58.124303 12785 init.cc:377] Logged in from keytab as oryx/127.5.61.2@KRBTEST.COM (short username oryx)
I20260809 20:17:58.125793 12785 webserver.cc:460] Webserver started at http://127.5.61.2:39987/ using document root <none> and password file <none>
I20260809 20:17:58.126477 12785 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:58.126622 12785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:58.126930 12785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:58.129876 12785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "07f1c4c3d4ee43b281427f86818468c1"
format_stamp: "Formatted at 2026-08-09 20:17:58 on dist-test-slave-6k22"
server_key: "367c5c904edf7143d7a55e0e402758dc"
server_key_iv: "d9887f5a33c732d9ce6495c5e4328523"
server_key_version: "encryptionkey@0"
I20260809 20:17:58.130614 12785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "07f1c4c3d4ee43b281427f86818468c1"
format_stamp: "Formatted at 2026-08-09 20:17:58 on dist-test-slave-6k22"
server_key: "367c5c904edf7143d7a55e0e402758dc"
server_key_iv: "d9887f5a33c732d9ce6495c5e4328523"
server_key_version: "encryptionkey@0"
I20260809 20:17:58.135747 12785 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260809 20:17:58.139185 12802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:58.140483 12785 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:17:58.140666 12785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "07f1c4c3d4ee43b281427f86818468c1"
format_stamp: "Formatted at 2026-08-09 20:17:58 on dist-test-slave-6k22"
server_key: "367c5c904edf7143d7a55e0e402758dc"
server_key_iv: "d9887f5a33c732d9ce6495c5e4328523"
server_key_version: "encryptionkey@0"
I20260809 20:17:58.140895 12785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:58.163141 12785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:58.167685 12785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:58.167991 12785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:58.168939 12785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:58.170209 12785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:58.170318 12785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:58.170460 12785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:58.170532 12785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:58.209542 12785 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:38017
I20260809 20:17:58.209668 12934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:38017 every 8 connection(s)
I20260809 20:17:58.211016 12785 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:17:58.214488 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 12785
I20260809 20:17:58.214745 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:17:58.215359 5364 external_mini_cluster.cc:1428] Setting key 1c5676ba64f55b69fd8f74246a0d72f6
Aug 09 20:17:58 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306678, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM
I20260809 20:17:58.237051 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:58.213750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:39509 (local address 127.5.61.62:40647)
0809 20:17:58.214278 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:58.214282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:58.215346 (+ 1064us) server_negotiation.cc:407] Connection header received
0809 20:17:58.219405 (+ 4059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:58.219412 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:58.219500 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:58.219633 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:58.221914 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:58.222613 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:58.225098 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:58.225548 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:58.225642 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:58.229555 (+ 3913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:58.229564 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:58.229572 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:58.229618 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:58.233519 (+ 3901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:58.234334 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:58.234341 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:58.234347 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:58.234519 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:58.234942 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:58.234950 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:58.234956 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:58.235361 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:58.235518 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:58.236277 (+ 759us) server_negotiation.cc:299] Negotiation successful
0809 20:17:58.236539 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":314,"threads_started":1}
I20260809 20:17:58.237684 12938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:58.214351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.5.61.2:39509)
0809 20:17:58.215189 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:58.215233 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:58.217176 (+ 1943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:58.220064 (+ 2888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:58.220077 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:58.220878 (+ 801us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:58.221721 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:58.221748 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:58.222807 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:58.222815 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:58.224962 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:58.224980 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:58.225765 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:58.225774 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:58.226078 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:58.226854 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:58.226876 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:58.229309 (+ 2433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:58.233760 (+ 4451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:58.233766 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:58.233777 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:58.234175 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:58.234633 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:58.234636 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:58.234638 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:58.234819 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:58.235485 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:58.235491 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:58.235874 (+ 383us) client_negotiation.cc:769] Sending connection context
0809 20:17:58.236116 (+ 242us) client_negotiation.cc:240] Negotiation successful
0809 20:17:58.236502 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":305,"threads_started":1}
I20260809 20:17:58.239331 12935 heartbeater.cc:347] Connected to a master server at 127.5.61.62:40647
I20260809 20:17:58.239741 12935 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:58.240661 12935 heartbeater.cc:511] Master 127.5.61.62:40647 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.5.61.3@KRBTEST.COM; defaulting to no policy
I20260809 20:17:58.242252 12559 ts_manager.cc:194] Registered new tserver with Master: 07f1c4c3d4ee43b281427f86818468c1 (127.5.61.2:38017)
I20260809 20:17:58.243086 12559 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.61.2@KRBTEST.COM'} at 127.5.61.2:39509
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.61.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.61.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.3.keytab.
Entry for principal oryx/127.5.61.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.3.keytab.
I20260809 20:17:58.272032 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:40647
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.3.keytab
--principal=oryx/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:58.604027 12941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:58.604311 12941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:58.604465 12941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:58.611793 12941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:58.611940 12941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:58.612107 12941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:17:58.621775 12941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.3.keytab
--principal=oryx/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:40647
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:58.623138 12941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:58.624616 12941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:58.638034 12947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:58.638078 12950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:58.638129 12948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:58.638980 12941 server_base.cc:1034] running on GCE node
I20260809 20:17:58.639506 12941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:58.640499 12941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:58.641703 12941 hybrid_clock.cc:648] HybridClock initialized: now 1786306678641660 us; error 29 us; skew 500 ppm
I20260809 20:17:58.642035 12941 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:58 dist-test-slave-6k22 krb5kdc[12503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306678, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:58.645105 12941 init.cc:377] Logged in from keytab as oryx/127.5.61.3@KRBTEST.COM (short username oryx)
I20260809 20:17:58.646519 12941 webserver.cc:460] Webserver started at http://127.5.61.3:34287/ using document root <none> and password file <none>
I20260809 20:17:58.647145 12941 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:58.647255 12941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:58.647497 12941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:58.650367 12941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "408146e0c09b428b8f9dd510aeecbdc6"
format_stamp: "Formatted at 2026-08-09 20:17:58 on dist-test-slave-6k22"
server_key: "d7dac5141b288c25652067abc2c5acc2"
server_key_iv: "1a0da5fa3c9f2e6cbf61155169b3d56f"
server_key_version: "encryptionkey@0"
I20260809 20:17:58.651085 12941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "408146e0c09b428b8f9dd510aeecbdc6"
format_stamp: "Formatted at 2026-08-09 20:17:58 on dist-test-slave-6k22"
server_key: "d7dac5141b288c25652067abc2c5acc2"
server_key_iv: "1a0da5fa3c9f2e6cbf61155169b3d56f"
server_key_version: "encryptionkey@0"
I20260809 20:17:58.656105 12941 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 20:17:58.659788 12958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:58.661434 12941 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:17:58.661597 12941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "408146e0c09b428b8f9dd510aeecbdc6"
format_stamp: "Formatted at 2026-08-09 20:17:58 on dist-test-slave-6k22"
server_key: "d7dac5141b288c25652067abc2c5acc2"
server_key_iv: "1a0da5fa3c9f2e6cbf61155169b3d56f"
server_key_version: "encryptionkey@0"
I20260809 20:17:58.661835 12941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:58.690773 12941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:58.695080 12941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:58.695386 12941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:58.696386 12941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:17:58.697751 12941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:17:58.697876 12941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:58.698016 12941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:17:58.698117 12941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:58.713372 12779 heartbeater.cc:503] Master 127.5.61.62:40647 was elected leader, sending a full tablet report...
I20260809 20:17:58.731819 12941 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:33467
I20260809 20:17:58.731851 13088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:33467 every 8 connection(s)
I20260809 20:17:58.733184 12941 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:17:58.735011 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 12941
I20260809 20:17:58.735230 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:17:58.735802 5364 external_mini_cluster.cc:1428] Setting key fdf0ef3e3102a60f4f0a4d81e8ef86e8
Aug 09 20:17:58 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306678, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM
I20260809 20:17:58.753628 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:58.736095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:39683 (local address 127.5.61.62:40647)
0809 20:17:58.736320 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:58.736327 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:58.737769 (+ 1442us) server_negotiation.cc:407] Connection header received
0809 20:17:58.740064 (+ 2295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:58.740070 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:58.740169 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:58.740301 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:58.741755 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:58.742473 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:58.744115 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:58.744538 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:58.744661 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:58.748064 (+ 3403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:58.748069 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:58.748072 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:58.748102 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:58.750966 (+ 2864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:58.751799 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:58.751803 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:58.751806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:58.751940 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:58.752297 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:58.752300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:58.752302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:58.752563 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:58.752669 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:58.753258 (+ 589us) server_negotiation.cc:299] Negotiation successful
0809 20:17:58.753423 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 20:17:58.754472 13091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:58.736756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.5.61.3:39683)
0809 20:17:58.737600 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:58.737644 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:58.739756 (+ 2112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:58.740533 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:58.740542 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:58.741033 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:58.741587 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:58.741602 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:58.742639 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:58.742645 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:58.743915 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:58.743927 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:58.744755 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:58.744760 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:58.744951 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:58.745688 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:58.745709 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:58.747892 (+ 2183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:58.751098 (+ 3206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:58.751104 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:58.751114 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:58.751681 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:58.752031 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:58.752034 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:58.752037 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:58.752201 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:58.752666 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:58.752671 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:58.752954 (+ 283us) client_negotiation.cc:769] Sending connection context
0809 20:17:58.753161 (+ 207us) client_negotiation.cc:240] Negotiation successful
0809 20:17:58.753483 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":385,"threads_started":1}
I20260809 20:17:58.755720 13089 heartbeater.cc:347] Connected to a master server at 127.5.61.62:40647
I20260809 20:17:58.756014 13089 heartbeater.cc:464] Registering TS with master...
I20260809 20:17:58.756716 13089 heartbeater.cc:511] Master 127.5.61.62:40647 requested a full tablet report, sending...
I20260809 20:17:58.758033 12559 ts_manager.cc:194] Registered new tserver with Master: 408146e0c09b428b8f9dd510aeecbdc6 (127.5.61.3:33467)
I20260809 20:17:58.758832 12559 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.61.3@KRBTEST.COM'} at 127.5.61.3:39683
I20260809 20:17:58.763377 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.61.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.61.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.61.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.61.keytab.
Entry for principal oryx/127.5.61.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.61.keytab.
I20260809 20:17:58.815212 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.61:42207
--webserver_interface=127.5.61.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.61.keytab
--principal=oryx/127.5.61.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:40647,127.5.61.61:42207
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:59.139125 13094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:59.139441 13094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:59.139663 13094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:59.146987 13094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:17:59.147107 13094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:59.147210 13094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:59.147307 13094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:17:59.147392 13094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:17:59.156669 13094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:40647,127.5.61.61:42207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.61.keytab
--principal=oryx/127.5.61.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.61:42207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.61.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
W20260809 20:17:59.157025 13094 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.5.61.62:40647,127.5.61.61:42207'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 20:17:59.158011 13094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:59.159451 13094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:59.171169 13099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:59.171131 13100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:59.171216 13102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:59.172161 13094 server_base.cc:1034] running on GCE node
I20260809 20:17:59.173028 13094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:59.174247 13094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:59.175489 13094 hybrid_clock.cc:648] HybridClock initialized: now 1786306679175441 us; error 52 us; skew 500 ppm
I20260809 20:17:59.175830 13094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:59 dist-test-slave-6k22 krb5kdc[12503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306679, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:59.179061 13094 init.cc:377] Logged in from keytab as oryx/127.5.61.61@KRBTEST.COM (short username oryx)
I20260809 20:17:59.180491 13094 webserver.cc:460] Webserver started at http://127.5.61.61:42033/ using document root <none> and password file <none>
I20260809 20:17:59.181151 13094 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:59.181284 13094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:59.181591 13094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:59.184468 13094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/data/instance:
uuid: "e272cdd4371e455c91ad74c33271afdd"
format_stamp: "Formatted at 2026-08-09 20:17:59 on dist-test-slave-6k22"
server_key: "94a0dd59541f29f38fafc2b53edd8d84"
server_key_iv: "0a4aaefe50368482e7cbb98ab8a54385"
server_key_version: "encryptionkey@0"
I20260809 20:17:59.185240 13094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/wal/instance:
uuid: "e272cdd4371e455c91ad74c33271afdd"
format_stamp: "Formatted at 2026-08-09 20:17:59 on dist-test-slave-6k22"
server_key: "94a0dd59541f29f38fafc2b53edd8d84"
server_key_iv: "0a4aaefe50368482e7cbb98ab8a54385"
server_key_version: "encryptionkey@0"
I20260809 20:17:59.190227 13094 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 20:17:59.193639 13111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:59.194975 13094 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 20:17:59.195149 13094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/wal
uuid: "e272cdd4371e455c91ad74c33271afdd"
format_stamp: "Formatted at 2026-08-09 20:17:59 on dist-test-slave-6k22"
server_key: "94a0dd59541f29f38fafc2b53edd8d84"
server_key_iv: "0a4aaefe50368482e7cbb98ab8a54385"
server_key_version: "encryptionkey@0"
I20260809 20:17:59.195353 13094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:59.208586 13094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:59.213764 13094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:59.214005 13094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:59.239276 13094 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.61:42207
I20260809 20:17:59.239287 13174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.61:42207 every 8 connection(s)
I20260809 20:17:59.241242 13094 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/data/info.pb
I20260809 20:17:59.245869 12935 heartbeater.cc:503] Master 127.5.61.62:40647 was elected leader, sending a full tablet report...
I20260809 20:17:59.246292 13175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:59.246732 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 13094
I20260809 20:17:59.257557 13175 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 20:17:59 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.61.61@KRBTEST.COM
I20260809 20:17:59.275552 13178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:59.249493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46014 (local address 127.5.61.61:42207)
0809 20:17:59.250374 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:59.250396 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:59.250453 (+ 57us) server_negotiation.cc:407] Connection header received
0809 20:17:59.253475 (+ 3022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:59.253502 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:59.253994 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:59.254458 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:59.256155 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.258542 (+ 2387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.260601 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.261555 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.261732 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:59.266839 (+ 5107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:59.266851 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:59.266868 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:59.266917 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:59.270895 (+ 3978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:59.271653 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:59.271666 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:59.271674 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:59.271866 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:59.272280 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:59.272285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:59.272289 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:59.273020 (+ 731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:59.273425 (+ 405us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:59.273776 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 20:17:59.274229 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":390,"threads_started":1}
Aug 09 20:17:59 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306679, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.61@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM
I20260809 20:17:59.286824 13180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:59.261009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55064 (local address 127.5.61.62:40647)
0809 20:17:59.261500 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:59.261506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:59.272353 (+ 10847us) server_negotiation.cc:407] Connection header received
0809 20:17:59.272562 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:59.272568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:59.272652 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:59.272770 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:59.274316 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.275457 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.277499 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.278120 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.278276 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:59.281265 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:59.281270 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:59.281273 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:59.281305 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:59.284060 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:59.284665 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:59.284670 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:59.284673 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:59.284840 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:59.285221 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:59.285227 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:59.285231 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:59.285583 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:59.285752 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:59.286186 (+ 434us) server_negotiation.cc:299] Negotiation successful
0809 20:17:59.286410 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":234,"threads_started":1}
I20260809 20:17:59.286854 13181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:59.261059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.0.0.1:55064)
0809 20:17:59.272209 (+ 11150us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:59.272259 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:59.272416 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:59.272817 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:59.272828 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:59.273764 (+ 936us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:59.274154 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:59.274170 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.275827 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.275834 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:59.277342 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:59.277359 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.278334 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.278340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:59.278534 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:59.279049 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:59.279066 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:59.281098 (+ 2032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:59.284169 (+ 3071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:59.284177 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:59.284181 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:59.284511 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:59.284950 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:59.284956 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:59.284958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:59.285098 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:59.285748 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:59.285755 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:59.285901 (+ 146us) client_negotiation.cc:769] Sending connection context
0809 20:17:59.286088 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 20:17:59.286394 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10922,"thread_start_us":10764,"threads_started":1}
I20260809 20:17:59.288605 13175 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 20:17:59 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306679, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.61@KRBTEST.COM for oryx/127.5.61.61@KRBTEST.COM
I20260809 20:17:59.300173 13181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:59.289214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:42207 (local address 127.0.0.1:46030)
0809 20:17:59.289399 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:59.289418 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:59.289521 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:59.289763 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:59.289766 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:59.290073 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:59.290201 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:59.290208 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.291164 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.291167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:59.292089 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:59.292100 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.292762 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.292766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:59.292889 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:59.293341 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:59.293357 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:59.295271 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:59.298111 (+ 2840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:59.298114 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:59.298117 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:59.298551 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:59.298882 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:59.298885 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:59.298887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:59.299008 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:59.299450 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:59.299453 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:59.299617 (+ 164us) client_negotiation.cc:769] Sending connection context
0809 20:17:59.299773 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 20:17:59.299968 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 20:17:59.300457 13178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:59.289337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46030 (local address 127.5.61.61:42207)
0809 20:17:59.289476 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:59.289480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:59.289517 (+ 37us) server_negotiation.cc:407] Connection header received
0809 20:17:59.289603 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:59.289606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:59.289656 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:59.289729 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:59.290314 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.291059 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.292214 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.292635 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.292730 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:59.295414 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:59.295419 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:59.295422 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:59.295447 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:59.298006 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:59.298650 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:59.298653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:59.298655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:59.298768 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:59.299084 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:59.299087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:59.299089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:59.299358 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:59.299462 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:59.299841 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 20:17:59.300112 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":57}
I20260809 20:17:59.304952 13175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd: Bootstrap starting.
I20260809 20:17:59.308111 13175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd: Neither blocks nor log segments found. Creating new log.
I20260809 20:17:59.309068 13175 log.cc:824] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd: Log is configured to *not* fsync() on all Append() calls
I20260809 20:17:59.311755 13175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd: No bootstrap required, opened a new log
I20260809 20:17:59.318496 13175 raft_consensus.cc:348] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } }
I20260809 20:17:59.318831 13175 raft_consensus.cc:374] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:17:59.318935 13175 raft_consensus.cc:729] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e272cdd4371e455c91ad74c33271afdd, State: Initialized, Role: FOLLOWER
I20260809 20:17:59.319479 13175 consensus_queue.cc:260] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } }
I20260809 20:17:59.320312 13183 sys_catalog.cc:455] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } } }
I20260809 20:17:59.320629 13183 sys_catalog.cc:458] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:17:59.321241 13175 sys_catalog.cc:564] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [sys.catalog]: configured and running, proceeding with master startup.
W20260809 20:17:59.329774 13195 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 20:17:59.329906 13195 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 20:17:59.333910 13094 master_runner.cc:428] Detected that this master e272cdd4371e455c91ad74c33271afdd is joining an existing cluster
I20260809 20:17:59.334010 13094 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.61.61:42207
I20260809 20:17:59.356741 13180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:59.338361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55078 (local address 127.5.61.62:40647)
0809 20:17:59.338561 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:59.338569 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:59.339251 (+ 682us) server_negotiation.cc:407] Connection header received
0809 20:17:59.339526 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:59.339531 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:59.339668 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:59.339751 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:59.341300 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.342377 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.344404 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.345058 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.345158 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:59.348023 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:59.348032 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:59.348039 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:59.348081 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:59.352095 (+ 4014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:59.353279 (+ 1184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:59.353287 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:59.353294 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:59.353478 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:59.354027 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:59.354035 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:59.354041 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:59.354444 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:59.354620 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:59.355755 (+ 1135us) server_negotiation.cc:299] Negotiation successful
0809 20:17:59.356036 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 20:17:59.358016 13201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:59.338417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.0.0.1:55078)
0809 20:17:59.339148 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:59.339168 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:59.339363 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:59.339995 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:59.340000 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:59.340686 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:59.341118 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:59.341132 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.342562 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.342567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:59.344221 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:59.344241 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.345450 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.345458 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:59.345627 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:59.346335 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:59.346371 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:59.347802 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:59.352288 (+ 4486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:59.352296 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:59.352303 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:59.352850 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:59.353630 (+ 780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:59.353639 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:59.353642 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:59.353871 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:59.354694 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:59.354700 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:59.354944 (+ 244us) client_negotiation.cc:769] Sending connection context
0809 20:17:59.355132 (+ 188us) client_negotiation.cc:240] Negotiation successful
0809 20:17:59.357493 (+ 2361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":409,"threads_started":1}
WARNING: no policy specified for oryx/127.5.61.60@KRBTEST.COM; defaulting to no policy
I20260809 20:17:59.361935 13178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:59.339639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46044 (local address 127.5.61.61:42207)
0809 20:17:59.339839 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:59.339847 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:59.346433 (+ 6586us) server_negotiation.cc:407] Connection header received
0809 20:17:59.346663 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:59.346668 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:59.346732 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:59.346810 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:59.348250 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.349364 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.351319 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.352075 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.352174 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:59.354587 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:59.354593 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:59.354597 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:59.354631 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:59.358441 (+ 3810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:59.359595 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:59.359600 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:59.359603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:59.359755 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:59.360231 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:59.360236 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:59.360239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:59.360606 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:59.360771 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:59.361295 (+ 524us) server_negotiation.cc:299] Negotiation successful
0809 20:17:59.361561 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.61.60@KRBTEST.COM" created.
I20260809 20:17:59.363118 13203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:59.339283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:42207 (local address 127.0.0.1:46044)
0809 20:17:59.346341 (+ 7058us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:59.346361 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:59.346523 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:59.347006 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:59.347011 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:59.347785 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:59.348102 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:59.348115 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.349609 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.349614 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:59.351171 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:59.351186 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.352461 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.352468 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:59.352627 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:59.353230 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:59.353255 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:59.354370 (+ 1115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:59.358912 (+ 4542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:59.358917 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:59.358921 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:59.359429 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:59.359869 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:59.359875 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:59.359916 (+ 41us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:59.360110 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:59.360774 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:59.360779 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:59.360948 (+ 169us) client_negotiation.cc:769] Sending connection context
0809 20:17:59.362300 (+ 1352us) client_negotiation.cc:240] Negotiation successful
0809 20:17:59.362608 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6924,"thread_start_us":6729,"threads_started":1}
Aug 09 20:17:59 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306676, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.62@KRBTEST.COM for oryx/127.5.61.61@KRBTEST.COM
I20260809 20:17:59.387095 13178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:59.365884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46050 (local address 127.5.61.61:42207)
0809 20:17:59.366044 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:59.366049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:59.373041 (+ 6992us) server_negotiation.cc:407] Connection header received
0809 20:17:59.373449 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:59.373457 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:59.373580 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:59.373695 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:59.375174 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.375972 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.377310 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.377778 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.377911 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:59.381310 (+ 3399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:59.381315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:59.381318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:59.381346 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:59.384308 (+ 2962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:59.384975 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:59.384980 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:59.384984 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:59.385113 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:59.385514 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:59.385519 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:59.385523 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:59.385789 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:59.385989 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:59.386460 (+ 471us) server_negotiation.cc:299] Negotiation successful
0809 20:17:59.386701 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 20:17:59.387151 13205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:59.365331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:42207 (local address 127.0.0.1:46050)
0809 20:17:59.372906 (+ 7575us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:59.372958 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:59.373207 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:59.373778 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:59.373788 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:59.374709 (+ 921us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:59.375023 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:59.375036 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.376116 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.376122 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:59.377134 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:59.377146 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.377943 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.377950 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:59.378148 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:59.378819 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:59.378836 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:59.381125 (+ 2289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:59.384458 (+ 3333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:59.384470 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:59.384477 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:59.384849 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:59.385243 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:59.385249 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:59.385253 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:59.385403 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:59.385999 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:59.386009 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:59.386183 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 20:17:59.386378 (+ 195us) client_negotiation.cc:240] Negotiation successful
0809 20:17:59.386723 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7310,"thread_start_us":7126,"threads_started":1}
I20260809 20:17:59.389511 12559 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.61.61:42207: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e272cdd4371e455c91ad74c33271afdd"
member_type: NON_VOTER
last_known_addr {
host: "127.5.61.61"
port: 42207
}
attrs {
promote: true
}
}
dest_uuid: "6798f13fda9446ddae84977ff493a57e"
cas_config_opid_index: -1
I20260809 20:17:59.390847 12559 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: NON_VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: true } }
I20260809 20:17:59.394551 13209 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER e272cdd4371e455c91ad74c33271afdd (127.5.61.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: NON_VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: true } } }
I20260809 20:17:59.395232 13144 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:17:59.400264 13144 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 1 FOLLOWER]: Refusing update from remote peer 6798f13fda9446ddae84977ff493a57e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 20:17:59.400980 13209 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.61.61:42207
I20260809 20:17:59.401559 13209 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: NON_VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.61.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.60.keytab.
Entry for principal oryx/127.5.61.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.60.keytab.
I20260809 20:17:59.401561 13210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6798f13fda9446ddae84977ff493a57e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: NON_VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: true } } }
I20260809 20:17:59.402899 13094 master.cc:537] Master@127.5.61.61:42207 shutting down...
I20260809 20:17:59.403975 13210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [sys.catalog]: This master's current role is: LEADER
I20260809 20:17:59.403966 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.60:44599
--webserver_interface=127.5.61.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.60.keytab
--principal=oryx/127.5.61.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:40647,127.5.61.61:42207,127.5.61.60:44599
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:17:59.405758 12543 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 20:17:59.419790 12543 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e -> Peer e272cdd4371e455c91ad74c33271afdd (127.5.61.61:42207): Couldn't send request to peer e272cdd4371e455c91ad74c33271afdd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 20:17:59.420241 13094 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:17:59.420665 13094 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:17:59.420898 13094 tablet_replica.cc:330] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd: stopping tablet replica
I20260809 20:17:59.427747 13094 master.cc:552] Master@127.5.61.61:42207 shutdown complete.
I20260809 20:17:59.427909 13094 master_runner.cc:305] Clearing existing system tablet
I20260809 20:17:59.428366 13094 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:59.428485 13094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:59.430810 13094 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 20:17:59.432412 13217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:59.432735 13094 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:59.433041 13094 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260809 20:17:59.433135 13094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/wal
uuid: "e272cdd4371e455c91ad74c33271afdd"
format_stamp: "Formatted at 2026-08-09 20:17:59 on dist-test-slave-6k22"
server_key: "94a0dd59541f29f38fafc2b53edd8d84"
server_key_iv: "0a4aaefe50368482e7cbb98ab8a54385"
server_key_version: "encryptionkey@0"
I20260809 20:17:59.437546 13094 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:17:59.443334 13094 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 20:17:59.443552 13094 log.cc:1196] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 20:17:59.444271 13094 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd: Deleting consensus metadata
I20260809 20:17:59.444566 13094 master_runner.cc:315] Copying system tablet from 127.5.61.62:40647
I20260809 20:17:59.447700 13094 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd: tablet copy: Beginning tablet copy session from remote peer at address 127.5.61.62:40647
I20260809 20:17:59.461493 13180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:59.448737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55080 (local address 127.5.61.62:40647)
0809 20:17:59.448976 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:59.448986 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:59.449522 (+ 536us) server_negotiation.cc:407] Connection header received
0809 20:17:59.449761 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:59.449768 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:59.449869 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:59.449991 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:59.450892 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.451753 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.453156 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.453633 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.453789 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:59.455678 (+ 1889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:59.455685 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:59.455690 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:59.455735 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:59.458601 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:59.459311 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:59.459318 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:59.459322 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:59.459458 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:59.459928 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:59.459939 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:59.459942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:59.460235 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:59.460410 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:59.460848 (+ 438us) server_negotiation.cc:299] Negotiation successful
0809 20:17:59.461056 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 20:17:59.461520 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:59.448703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.0.0.1:55080)
0809 20:17:59.449398 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:59.449416 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:59.449592 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:59.450018 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:59.450023 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:59.450468 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:59.450724 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:59.450733 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.451915 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.451922 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:59.452985 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:59.452998 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.453804 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.453811 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:59.454016 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:59.454687 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:59.454706 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:59.455439 (+ 733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:59.458772 (+ 3333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:59.458778 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:59.458782 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:59.459168 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:59.459633 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:59.459655 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:59.459658 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:59.459794 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:59.460374 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:59.460387 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:59.460577 (+ 190us) client_negotiation.cc:769] Sending connection context
0809 20:17:59.460748 (+ 171us) client_negotiation.cc:240] Negotiation successful
0809 20:17:59.461051 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":446,"threads_started":1}
I20260809 20:17:59.462809 12582 tablet_copy_service.cc:140] P 6798f13fda9446ddae84977ff493a57e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e272cdd4371e455c91ad74c33271afdd ({username='oryx', principal='oryx/127.5.61.61@KRBTEST.COM'} at 127.0.0.1:55080)
I20260809 20:17:59.463230 12582 tablet_copy_service.cc:161] P 6798f13fda9446ddae84977ff493a57e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e272cdd4371e455c91ad74c33271afdd at {username='oryx', principal='oryx/127.5.61.61@KRBTEST.COM'} at 127.0.0.1:55080: session id = e272cdd4371e455c91ad74c33271afdd-00000000000000000000000000000000
I20260809 20:17:59.467051 12582 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 20:17:59.469467 13094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 20:17:59.474515 13094 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd: tablet copy: Starting download of 0 data blocks...
I20260809 20:17:59.474964 13094 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd: tablet copy: Starting download of 1 WAL segments...
I20260809 20:17:59.478281 13094 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 20:17:59.481748 12582 tablet_copy_service.cc:342] P 6798f13fda9446ddae84977ff493a57e: Request end of tablet copy session e272cdd4371e455c91ad74c33271afdd-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.61.61@KRBTEST.COM'} at 127.0.0.1:55080
I20260809 20:17:59.481992 12582 tablet_copy_service.cc:433] P 6798f13fda9446ddae84977ff493a57e: ending tablet copy session e272cdd4371e455c91ad74c33271afdd-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e272cdd4371e455c91ad74c33271afdd
I20260809 20:17:59.493415 13094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:59.497779 13225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:59.501104 13226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:59.510740 13094 server_base.cc:1034] running on GCE node
W20260809 20:17:59.511891 13228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:59.512341 13094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:59.513223 13094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:59.514441 13094 hybrid_clock.cc:648] HybridClock initialized: now 1786306679514341 us; error 71 us; skew 500 ppm
I20260809 20:17:59.514672 13094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:59 dist-test-slave-6k22 krb5kdc[12503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306679, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:59.516660 13094 init.cc:377] Logged in from keytab as oryx/127.5.61.61@KRBTEST.COM (short username oryx)
I20260809 20:17:59.517985 13094 webserver.cc:460] Webserver started at http://127.5.61.61:34819/ using document root <none> and password file <none>
I20260809 20:17:59.518311 13094 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:59.518433 13094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:59.520838 13094 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 20:17:59.522608 13236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:59.523059 13094 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260809 20:17:59.523222 13094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/wal
uuid: "e272cdd4371e455c91ad74c33271afdd"
format_stamp: "Formatted at 2026-08-09 20:17:59 on dist-test-slave-6k22"
server_key: "94a0dd59541f29f38fafc2b53edd8d84"
server_key_iv: "0a4aaefe50368482e7cbb98ab8a54385"
server_key_version: "encryptionkey@0"
I20260809 20:17:59.523478 13094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:59.534466 13094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:59.537850 13094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:59.561756 13094 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.61:42207
I20260809 20:17:59.561779 13295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.61:42207 every 8 connection(s)
I20260809 20:17:59.562671 13094 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-1/data/info.pb
I20260809 20:17:59.570106 13296 sys_catalog.cc:263] Verifying existing consensus state
I20260809 20:17:59.575624 13296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd: Bootstrap starting.
I20260809 20:17:59.590154 13296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 20:17:59.590675 13296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd: Bootstrap complete.
I20260809 20:17:59.591634 13296 raft_consensus.cc:348] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: NON_VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: true } }
I20260809 20:17:59.591871 13296 raft_consensus.cc:729] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e272cdd4371e455c91ad74c33271afdd, State: Initialized, Role: LEARNER
I20260809 20:17:59.592083 13296 consensus_queue.cc:260] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: NON_VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: true } }
I20260809 20:17:59.592876 13299 sys_catalog.cc:455] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: NON_VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: true } } }
I20260809 20:17:59.593180 13296 sys_catalog.cc:564] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:17:59.593290 13299 sys_catalog.cc:458] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [sys.catalog]: This master's current role is: LEARNER
I20260809 20:17:59.610086 13310 catalog_manager.cc:1259] Loaded cluster ID: 19ac0cc23316463f8eba2700c4b6ac78
I20260809 20:17:59.610194 13310 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd: loading cluster ID for follower catalog manager: success
I20260809 20:17:59.612841 13310 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd: acquiring CA information for follower catalog manager: success
I20260809 20:17:59.614992 13310 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 20:17:59.761251 13089 heartbeater.cc:503] Master 127.5.61.62:40647 was elected leader, sending a full tablet report...
W20260809 20:17:59.864630 13215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:17:59.865041 13215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:17:59.865221 13215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:17:59.872489 13215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:17:59.872632 13215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:17:59.872762 13215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:17:59.872869 13215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:17:59.872947 13215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:17:59.883018 13215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:40647,127.5.61.61:42207,127.5.61.60:44599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.60.keytab
--principal=oryx/127.5.61.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.60:44599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.61.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:17:59.884407 13215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:17:59.885730 13215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:17:59.897941 13316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:59.897890 13320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:17:59.897853 13317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:17:59.898768 13215 server_base.cc:1034] running on GCE node
I20260809 20:17:59.899434 13215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:17:59.900815 13215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:17:59.902027 13215 hybrid_clock.cc:648] HybridClock initialized: now 1786306679901969 us; error 39 us; skew 500 ppm
I20260809 20:17:59.902529 13215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:17:59 dist-test-slave-6k22 krb5kdc[12503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306679, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:17:59.912563 13215 init.cc:377] Logged in from keytab as oryx/127.5.61.60@KRBTEST.COM (short username oryx)
I20260809 20:17:59.914242 13215 webserver.cc:460] Webserver started at http://127.5.61.60:38617/ using document root <none> and password file <none>
I20260809 20:17:59.915004 13215 fs_manager.cc:359] Metadata directory not provided
I20260809 20:17:59.915155 13215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:17:59.915465 13215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:17:59.918963 13215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/data/instance:
uuid: "32de6f510b03403ead9eca45eae81f33"
format_stamp: "Formatted at 2026-08-09 20:17:59 on dist-test-slave-6k22"
server_key: "3ad8f6614772f1394346897daef93e58"
server_key_iv: "5e99e60a0a2fa261231517989c222af4"
server_key_version: "encryptionkey@0"
I20260809 20:17:59.919938 13215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/wal/instance:
uuid: "32de6f510b03403ead9eca45eae81f33"
format_stamp: "Formatted at 2026-08-09 20:17:59 on dist-test-slave-6k22"
server_key: "3ad8f6614772f1394346897daef93e58"
server_key_iv: "5e99e60a0a2fa261231517989c222af4"
server_key_version: "encryptionkey@0"
I20260809 20:17:59.924960 13329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:59.917069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46064 (local address 127.5.61.61:42207)
0809 20:17:59.918146 (+ 1077us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:59.918150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:59.918169 (+ 19us) server_negotiation.cc:407] Connection header received
0809 20:17:59.918227 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:59.918231 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:59.918294 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:59.918490 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:17:59.919089 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.920332 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.922713 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.923795 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.923897 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:59.924125 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:59.924410 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 20:17:59.924570 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":972,"thread_start_us":800,"threads_started":1}
I20260809 20:17:59.924970 13327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:59.916850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:42207 (local address 127.0.0.1:46064)
0809 20:17:59.917342 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:59.917388 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:59.917568 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:59.918569 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:59.918575 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:59.918610 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:17:59.918843 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:59.918861 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.920492 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.920497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:59.922538 (+ 2041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:59.922557 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.923955 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.923963 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:59.924159 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:59.924216 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 20:17:59.924352 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 20:17:59.924513 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":217,"threads_started":1}
I20260809 20:17:59.926195 13268 raft_consensus.cc:1203] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260809 20:17:59.926389 13215 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260809 20:17:59.926751 13299 sys_catalog.cc:455] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6798f13fda9446ddae84977ff493a57e. Latest consensus state: current_term: 1 leader_uuid: "6798f13fda9446ddae84977ff493a57e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: NON_VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: true } } }
I20260809 20:17:59.927007 13299 sys_catalog.cc:458] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [sys.catalog]: This master's current role is: LEARNER
I20260809 20:17:59.929994 13332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:17:59.931216 13215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 20:17:59.931419 13215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/wal
uuid: "32de6f510b03403ead9eca45eae81f33"
format_stamp: "Formatted at 2026-08-09 20:17:59 on dist-test-slave-6k22"
server_key: "3ad8f6614772f1394346897daef93e58"
server_key_iv: "5e99e60a0a2fa261231517989c222af4"
server_key_version: "encryptionkey@0"
I20260809 20:17:59.931661 13215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:17:59.943710 13215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:17:59.947522 13215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:17:59.947845 13215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:17:59.966307 13215 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.60:44599
I20260809 20:17:59.966372 13388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.60:44599 every 8 connection(s)
I20260809 20:17:59.967722 13215 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/data/info.pb
I20260809 20:17:59.970819 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 13215
I20260809 20:17:59.971238 13389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:17:59.981351 13389 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 20:17:59 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.61.60@KRBTEST.COM
Aug 09 20:17:59 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306679, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.60@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM
I20260809 20:17:59.996517 13392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:59.973042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52956 (local address 127.5.61.60:44599)
0809 20:17:59.973833 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:59.973853 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:59.973904 (+ 51us) server_negotiation.cc:407] Connection header received
0809 20:17:59.976092 (+ 2188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:59.976118 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:59.976648 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:59.977208 (+ 560us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:59.978531 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.980529 (+ 1998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.981815 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.982855 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.983035 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:59.988250 (+ 5215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:59.988261 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:59.988271 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:59.988317 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:59.991682 (+ 3365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:59.992381 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:59.992389 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:59.992397 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:59.992610 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:59.993124 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:59.993132 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:59.993163 (+ 31us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:59.993908 (+ 745us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:59.994196 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:59.994614 (+ 418us) server_negotiation.cc:299] Negotiation successful
0809 20:17:59.995030 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":336,"threads_started":1}
I20260809 20:18:00.000442 13393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:59.984254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.0.0.1:55096)
0809 20:17:59.985626 (+ 1372us) negotiation.cc:107] Waiting for socket to connect
0809 20:17:59.985666 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 20:17:59.985789 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:17:59.986198 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:17:59.986203 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:17:59.986779 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:17:59.987014 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:59.987022 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.988211 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.988221 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:59.989522 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:17:59.989534 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.990501 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.990508 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:17:59.990707 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:59.991336 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:17:59.991371 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:17:59.993666 (+ 2295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:17:59.997533 (+ 3867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:59.997542 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:59.997548 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:59.998012 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:59.998434 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:17:59.998437 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:17:59.998439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:17:59.998550 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:17:59.999127 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:17:59.999132 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:17:59.999374 (+ 242us) client_negotiation.cc:769] Sending connection context
0809 20:17:59.999675 (+ 301us) client_negotiation.cc:240] Negotiation successful
0809 20:17:59.999999 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":984,"thread_start_us":335,"threads_started":1}
I20260809 20:18:00.000541 13394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:17:59.984396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55096 (local address 127.5.61.62:40647)
0809 20:17:59.985650 (+ 1254us) server_negotiation.cc:206] Beginning negotiation
0809 20:17:59.985656 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:17:59.985762 (+ 106us) server_negotiation.cc:407] Connection header received
0809 20:17:59.985915 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:17:59.985922 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:17:59.986014 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:17:59.986142 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:17:59.987172 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.988044 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.989710 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:17:59.990292 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:17:59.990445 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:17:59.993902 (+ 3457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:17:59.993908 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:17:59.993913 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:17:59.993955 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:17:59.997398 (+ 3443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:59.998137 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:59.998141 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:59.998144 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:59.998311 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:17:59.998680 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:17:59.998685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:17:59.998688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:17:59.999009 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:17:59.999173 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 20:17:59.999777 (+ 604us) server_negotiation.cc:299] Negotiation successful
0809 20:18:00.000062 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1114,"thread_start_us":980,"threads_started":1}
I20260809 20:18:00.001607 13389 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 20:18:00 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306679, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.60@KRBTEST.COM for oryx/127.5.61.61@KRBTEST.COM
I20260809 20:18:00.014156 13393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:00.002182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:42207 (local address 127.0.0.1:46076)
0809 20:18:00.002393 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:00.002409 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:00.002482 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:00.002781 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:00.002784 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:00.003060 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:00.003182 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:00.003189 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.004514 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.004521 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:00.005525 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:00.005537 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.006213 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.006217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:00.006364 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:00.006877 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:00.006894 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:00.008952 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:00.012132 (+ 3180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:00.012135 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:00.012138 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:00.012475 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:00.012808 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:00.012812 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:00.012814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:00.012924 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:00.013490 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:00.013493 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:00.013612 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 20:18:00.013723 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 20:18:00.013928 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 20:18:00.014261 13329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:00.002377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46076 (local address 127.5.61.61:42207)
0809 20:18:00.002538 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:00.002542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:00.002557 (+ 15us) server_negotiation.cc:407] Connection header received
0809 20:18:00.002604 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:00.002607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:00.002661 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:00.002740 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:00.003317 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.004351 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.005653 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.006055 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.006210 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:00.009141 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:00.009145 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:00.009148 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:00.009179 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:00.012013 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:00.012575 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:00.012579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:00.012582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:00.012716 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:00.013028 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:00.013032 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:00.013033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:00.013368 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:00.013528 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:00.013826 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 20:18:00.013991 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 20:18:00.015206 13389 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.60" port: 44599 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 20:18:00 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306679, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.60@KRBTEST.COM for oryx/127.5.61.60@KRBTEST.COM
I20260809 20:18:00.027458 13393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:00.016120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:44599 (local address 127.0.0.1:52960)
0809 20:18:00.016321 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:00.016338 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:00.016427 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:00.016691 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:00.016694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:00.016993 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:00.017126 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:00.017132 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.017999 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.018002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:00.018827 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:00.018837 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.019801 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.019807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:00.020095 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:00.020602 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:00.020617 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:00.022435 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:00.025559 (+ 3124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:00.025562 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:00.025565 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:00.025896 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:00.026193 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:00.026196 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:00.026198 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:00.026347 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:00.026801 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:00.026804 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:00.026910 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 20:18:00.027017 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 20:18:00.027188 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 20:18:00.027659 13392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:00.016183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52960 (local address 127.5.61.60:44599)
0809 20:18:00.016348 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:00.016351 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:00.016400 (+ 49us) server_negotiation.cc:407] Connection header received
0809 20:18:00.016511 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:00.016514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:00.016583 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:00.016658 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:00.017236 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.017899 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.018941 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.019538 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.019742 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:00.022640 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:00.022650 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:00.022653 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:00.022680 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:00.025436 (+ 2756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:00.025989 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:00.025993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:00.025995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:00.026093 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:00.026435 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:00.026438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:00.026440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:00.026705 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:00.026810 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:00.027096 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 20:18:00.027332 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":52}
I20260809 20:18:00.032344 13389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33: Bootstrap starting.
I20260809 20:18:00.035609 13389 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:00.036791 13389 log.cc:824] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33: Log is configured to *not* fsync() on all Append() calls
I20260809 20:18:00.039259 13389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33: No bootstrap required, opened a new log
I20260809 20:18:00.046550 13389 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 44599 } }
I20260809 20:18:00.046882 13389 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:00.047015 13389 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32de6f510b03403ead9eca45eae81f33, State: Initialized, Role: FOLLOWER
I20260809 20:18:00.047528 13389 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 44599 } }
I20260809 20:18:00.048429 13396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 44599 } } }
I20260809 20:18:00.048722 13396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:18:00.049355 13389 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 20:18:00.058327 13408 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 20:18:00.058525 13408 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 20:18:00.062834 13215 master_runner.cc:428] Detected that this master 32de6f510b03403ead9eca45eae81f33 is joining an existing cluster
I20260809 20:18:00.062947 13215 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.61.60:44599
I20260809 20:18:00.089010 13394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:00.067209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55102 (local address 127.5.61.62:40647)
0809 20:18:00.067377 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:00.067384 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:00.071543 (+ 4159us) server_negotiation.cc:407] Connection header received
0809 20:18:00.072061 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:00.072067 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:00.072155 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:00.072216 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:00.073658 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.074842 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.079015 (+ 4173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.079704 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.079793 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:00.082467 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:00.082474 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:00.082478 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:00.082516 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:00.086031 (+ 3515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:00.086806 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:00.086811 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:00.086824 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:00.086985 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:00.087443 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:00.087449 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:00.087453 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:00.087791 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:00.087913 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:00.088452 (+ 539us) server_negotiation.cc:299] Negotiation successful
0809 20:18:00.088669 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 20:18:00.089133 13419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:00.067336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.0.0.1:55102)
0809 20:18:00.071442 (+ 4106us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:00.071465 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:00.071915 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:00.072432 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:00.072436 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:00.073160 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:00.073506 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:00.073517 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.075041 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.075046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:00.076709 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:00.076723 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.080093 (+ 3370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.080099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:00.080276 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:00.080963 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:00.080993 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:00.082107 (+ 1114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:00.086209 (+ 4102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:00.086214 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:00.086217 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:00.086674 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:00.087125 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:00.087129 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:00.087132 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:00.087310 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:00.087991 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:00.087994 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:00.088179 (+ 185us) client_negotiation.cc:769] Sending connection context
0809 20:18:00.088340 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 20:18:00.088648 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3974,"thread_start_us":9633,"threads_started":1}
I20260809 20:18:00.102682 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 12626
I20260809 20:18:00.106184 13392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:00.067184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52972 (local address 127.5.61.60:44599)
0809 20:18:00.067377 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:00.067384 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:00.077339 (+ 9955us) server_negotiation.cc:407] Connection header received
0809 20:18:00.077505 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:00.077510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:00.077572 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:00.077630 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:00.082263 (+ 4633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.083998 (+ 1735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.094445 (+ 10447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.095156 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.095244 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:00.097769 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:00.097785 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:00.097789 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:00.097826 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:00.102141 (+ 4315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:00.103098 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:00.103105 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:00.103111 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:00.103290 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:00.103789 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:00.103797 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:00.103802 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:00.104340 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:00.104550 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:00.105269 (+ 719us) server_negotiation.cc:299] Negotiation successful
0809 20:18:00.105548 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 20:18:00.113129 13329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:00.077558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46078 (local address 127.5.61.61:42207)
0809 20:18:00.077746 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:00.077753 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:00.089468 (+ 11715us) server_negotiation.cc:407] Connection header received
0809 20:18:00.089696 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:00.089700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:00.089763 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:00.089823 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:00.090712 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.091486 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.092867 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.093506 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.093608 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:00.096167 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:00.096174 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:00.096179 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:00.096218 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:00.099782 (+ 3564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:00.100775 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:00.100780 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:00.100783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:00.100954 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:00.111136 (+ 10182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:00.111142 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:00.111145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:00.111528 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:00.111696 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:00.112498 (+ 802us) server_negotiation.cc:299] Negotiation successful
0809 20:18:00.112751 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 20:18:00.114372 13419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:00.077892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:42207 (local address 127.0.0.1:46078)
0809 20:18:00.089375 (+ 11483us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:00.089392 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:00.089550 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:00.090055 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:00.090058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:00.090421 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:00.090560 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:00.090568 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.091670 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.091676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:00.092632 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:00.092648 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.093862 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.093869 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:00.094172 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:00.094770 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:00.094792 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:00.095960 (+ 1168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:00.099987 (+ 4027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:00.099991 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:00.099995 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:00.100637 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:00.101111 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:00.101115 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:00.101118 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:00.110719 (+ 9601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:00.112029 (+ 1310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:00.112035 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:00.112175 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 20:18:00.113647 (+ 1472us) client_negotiation.cc:240] Negotiation successful
0809 20:18:00.113975 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11376,"mutex_wait_us":58,"thread_start_us":15060,"threads_started":1}
Aug 09 20:18:00 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306676, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.62@KRBTEST.COM for oryx/127.5.61.60@KRBTEST.COM
I20260809 20:18:00.117373 13417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:00.067804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:44599 (local address 127.0.0.1:52972)
0809 20:18:00.077258 (+ 9454us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:00.077278 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:00.077407 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:00.077797 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:00.077802 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:00.078526 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:00.078864 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:00.078875 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.092403 (+ 13528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.092412 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:00.094299 (+ 1887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:00.094317 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.095588 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.095595 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:00.095788 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:00.096355 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:00.096375 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:00.097583 (+ 1208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:00.102319 (+ 4736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:00.102326 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:00.102332 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:00.102964 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:00.103436 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:00.103443 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:00.103448 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:00.103662 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:00.104625 (+ 963us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:00.104632 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:00.104857 (+ 225us) client_negotiation.cc:769] Sending connection context
0809 20:18:00.116621 (+ 11764us) client_negotiation.cc:240] Negotiation successful
0809 20:18:00.116926 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9333,"mutex_wait_us":236,"thread_start_us":3757,"threads_started":1}
I20260809 20:18:00.124517 13392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:00.099442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52988 (local address 127.5.61.60:44599)
0809 20:18:00.106437 (+ 6995us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:00.106445 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:00.106467 (+ 22us) server_negotiation.cc:407] Connection header received
0809 20:18:00.106579 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:00.106587 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:00.106836 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:00.108461 (+ 1625us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:00.108677 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.110101 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.112198 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.112925 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.116463 (+ 3538us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:00.118294 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:00.118300 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:00.118304 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:00.118340 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:00.121714 (+ 3374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:00.122399 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:00.122404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:00.122407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:00.122520 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:00.122911 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:00.122916 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:00.122919 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:00.123312 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:00.123447 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:00.123937 (+ 490us) server_negotiation.cc:299] Negotiation successful
0809 20:18:00.124142 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6876,"thread_start_us":13624,"threads_started":1}
I20260809 20:18:00.124580 13327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:00.098723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:44599 (local address 127.0.0.1:52988)
0809 20:18:00.099112 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:00.099133 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:00.099233 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:00.107048 (+ 7815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:00.107056 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:00.107687 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:00.107975 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:00.107992 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.110310 (+ 2318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.110319 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:00.112021 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:00.112036 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.113123 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.113129 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:00.113316 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:00.114077 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:00.114107 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:00.118087 (+ 3980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:00.121870 (+ 3783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:00.121877 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:00.121881 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:00.122258 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:00.122641 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:00.122648 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:00.122651 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:00.122782 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:00.123416 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:00.123421 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:00.123637 (+ 216us) client_negotiation.cc:769] Sending connection context
0809 20:18:00.123776 (+ 139us) client_negotiation.cc:240] Negotiation successful
0809 20:18:00.124116 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254}
Aug 09 20:18:00 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.61.1@KRBTEST.COM
I20260809 20:18:00.126232 12559 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.61.60:44599: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "32de6f510b03403ead9eca45eae81f33"
member_type: NON_VOTER
last_known_addr {
host: "127.5.61.60"
port: 44599
}
attrs {
promote: true
}
}
dest_uuid: "6798f13fda9446ddae84977ff493a57e"
cas_config_opid_index: 5
I20260809 20:18:00.127374 12559 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: NON_VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: true } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: NON_VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: true } }
I20260809 20:18:00.129431 13210 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 32de6f510b03403ead9eca45eae81f33 (127.5.61.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: NON_VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: true } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: NON_VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: true } } }
I20260809 20:18:00.134485 13210 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.61.60:44599
I20260809 20:18:00.135129 13215 master.cc:537] Master@127.5.61.60:44599 shutting down...
I20260809 20:18:00.135277 13422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:00.104487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39348 (local address 127.5.61.1:37061)
0809 20:18:00.111987 (+ 7500us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:00.111996 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:00.112027 (+ 31us) server_negotiation.cc:407] Connection header received
0809 20:18:00.112097 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:00.112106 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:00.112416 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:00.115451 (+ 3035us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:00.115750 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.117266 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.119380 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.120553 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.120689 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:00.126371 (+ 5682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:00.126385 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:00.126403 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:00.126452 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:00.130510 (+ 4058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:00.131429 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:00.131437 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:00.131441 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:00.131735 (+ 294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:00.132202 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:00.132207 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:00.132210 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:00.132630 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:00.134187 (+ 1557us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:00.134471 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 20:18:00.134747 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7304,"thread_start_us":7222,"threads_started":1}
I20260809 20:18:00.135944 13268 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 1 LEARNER]: Refusing update from remote peer 6798f13fda9446ddae84977ff493a57e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 20:18:00.136838 12773 generic_service.cc:196] Checking for leaks (request via RPC)
W20260809 20:18:00.142736 12545 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e -> Peer 32de6f510b03403ead9eca45eae81f33 (127.5.61.60:44599): Couldn't send request to peer 32de6f510b03403ead9eca45eae81f33. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 20:18:00.159931 13424 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: NON_VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 20:18:00.160799 13423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6798f13fda9446ddae84977ff493a57e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: NON_VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: true } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: NON_VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: true } } }
I20260809 20:18:00.161334 13423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [sys.catalog]: This master's current role is: LEADER
I20260809 20:18:00.163036 13268 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 32de6f510b03403ead9eca45eae81f33 (127.5.61.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: NON_VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: true } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: NON_VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: true } } }
I20260809 20:18:00.166510 13299 sys_catalog.cc:455] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6798f13fda9446ddae84977ff493a57e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: NON_VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: true } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: NON_VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: true } } }
I20260809 20:18:00.166796 13299 sys_catalog.cc:458] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [sys.catalog]: This master's current role is: LEARNER
I20260809 20:18:00.167804 13424 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e: attempting to promote NON_VOTER e272cdd4371e455c91ad74c33271afdd to VOTER
I20260809 20:18:00.168664 13424 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: NON_VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: true } }
I20260809 20:18:00.170713 13268 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 1 LEARNER]: Refusing update from remote peer 6798f13fda9446ddae84977ff493a57e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260809 20:18:00.171310 12545 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e -> Peer 32de6f510b03403ead9eca45eae81f33 (127.5.61.60:44599): Couldn't send request to peer 32de6f510b03403ead9eca45eae81f33. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 20:18:00.171399 13424 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 20:18:00.173590 13423 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e272cdd4371e455c91ad74c33271afdd (127.5.61.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: NON_VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: true } } }
I20260809 20:18:00.174357 13268 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e272cdd4371e455c91ad74c33271afdd (127.5.61.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: NON_VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: true } } }
I20260809 20:18:00.178576 13215 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 20:18:00.178656 13299 sys_catalog.cc:455] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6798f13fda9446ddae84977ff493a57e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: NON_VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: true } } }
I20260809 20:18:00.178922 13299 sys_catalog.cc:458] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:18:00.179128 13215 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 20:18:00.179387 13215 tablet_replica.cc:330] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33: stopping tablet replica
I20260809 20:18:00.181180 13210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6798f13fda9446ddae84977ff493a57e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: NON_VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: true } } }
I20260809 20:18:00.181411 13210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [sys.catalog]: This master's current role is: LEADER
I20260809 20:18:00.181931 13423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6798f13fda9446ddae84977ff493a57e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: NON_VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: true } } }
I20260809 20:18:00.182472 13423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [sys.catalog]: This master's current role is: LEADER
I20260809 20:18:00.197207 13215 master.cc:552] Master@127.5.61.60:44599 shutdown complete.
I20260809 20:18:00.197296 13215 master_runner.cc:305] Clearing existing system tablet
I20260809 20:18:00.197705 13215 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:00.197855 13215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:00.200941 13215 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 20:18:00.202603 13431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:00.202973 13215 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:00.203222 13215 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260809 20:18:00.203325 13215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/wal
uuid: "32de6f510b03403ead9eca45eae81f33"
format_stamp: "Formatted at 2026-08-09 20:17:59 on dist-test-slave-6k22"
server_key: "3ad8f6614772f1394346897daef93e58"
server_key_iv: "5e99e60a0a2fa261231517989c222af4"
server_key_version: "encryptionkey@0"
I20260809 20:18:00.208439 13215 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:18:00.215602 13215 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 20:18:00.215761 13215 log.cc:1196] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 20:18:00.216228 13215 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33: Deleting consensus metadata
I20260809 20:18:00.216477 13215 master_runner.cc:315] Copying system tablet from 127.5.61.62:40647
I20260809 20:18:00.224745 13215 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33: tablet copy: Beginning tablet copy session from remote peer at address 127.5.61.62:40647
I20260809 20:18:00.240355 13394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:00.225550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55112 (local address 127.5.61.62:40647)
0809 20:18:00.225779 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:00.225786 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:00.226183 (+ 397us) server_negotiation.cc:407] Connection header received
0809 20:18:00.226349 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:00.226355 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:00.226449 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:00.226572 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:00.227621 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.228757 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.230718 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.231376 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.231525 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:00.233825 (+ 2300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:00.233830 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:00.233835 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:00.233871 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:00.237322 (+ 3451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:00.238106 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:00.238112 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:00.238116 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:00.238277 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:00.238718 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:00.238723 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:00.238727 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:00.239104 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:00.239262 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:00.239755 (+ 493us) server_negotiation.cc:299] Negotiation successful
0809 20:18:00.239990 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 20:18:00.240376 13438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:00.225516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.0.0.1:55112)
0809 20:18:00.226061 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:00.226084 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:00.226223 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:00.226627 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:00.226633 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:00.227159 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:00.227420 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:00.227433 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.228941 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.228948 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:00.230556 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:00.230570 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.231554 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.231561 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:00.231768 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:00.232500 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:00.232523 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:00.233595 (+ 1072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:00.237471 (+ 3876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:00.237477 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:00.237481 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:00.237981 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:00.238418 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:00.238423 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:00.238427 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:00.238597 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:00.239247 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:00.239253 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:00.239433 (+ 180us) client_negotiation.cc:769] Sending connection context
0809 20:18:00.239612 (+ 179us) client_negotiation.cc:240] Negotiation successful
0809 20:18:00.239917 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":289,"threads_started":1}
I20260809 20:18:00.241284 12582 tablet_copy_service.cc:140] P 6798f13fda9446ddae84977ff493a57e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 32de6f510b03403ead9eca45eae81f33 ({username='oryx', principal='oryx/127.5.61.60@KRBTEST.COM'} at 127.0.0.1:55112)
I20260809 20:18:00.241458 12582 tablet_copy_service.cc:161] P 6798f13fda9446ddae84977ff493a57e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 32de6f510b03403ead9eca45eae81f33 at {username='oryx', principal='oryx/127.5.61.60@KRBTEST.COM'} at 127.0.0.1:55112: session id = 32de6f510b03403ead9eca45eae81f33-00000000000000000000000000000000
I20260809 20:18:00.245362 12582 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 20:18:00.247134 13215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 20:18:00.255020 13215 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33: tablet copy: Starting download of 0 data blocks...
I20260809 20:18:00.255366 13215 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33: tablet copy: Starting download of 1 WAL segments...
I20260809 20:18:00.258018 13215 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 20:18:00.262497 12582 tablet_copy_service.cc:342] P 6798f13fda9446ddae84977ff493a57e: Request end of tablet copy session 32de6f510b03403ead9eca45eae81f33-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.61.60@KRBTEST.COM'} at 127.0.0.1:55112
I20260809 20:18:00.262681 12582 tablet_copy_service.cc:433] P 6798f13fda9446ddae84977ff493a57e: ending tablet copy session 32de6f510b03403ead9eca45eae81f33-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 32de6f510b03403ead9eca45eae81f33
I20260809 20:18:00.273226 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 12626
I20260809 20:18:00.279510 13215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:00.282866 13439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:00.288694 13440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:00.299873 13215 server_base.cc:1034] running on GCE node
W20260809 20:18:00.300977 13443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:00.301636 13215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:00.301826 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:37061
--local_ip_for_outbound_sockets=127.5.61.1
--tserver_master_addrs=127.5.61.62:40647
--webserver_port=39695
--webserver_interface=127.5.61.1
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.1.keytab
--principal=oryx/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.61.62:40647,127.5.61.61:42207
--tserver_master_addrs=127.5.61.62:40647,127.5.61.61:42207,127.5.61.60:44599 with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 20:18:00.302973 13215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:00.304304 13215 hybrid_clock.cc:648] HybridClock initialized: now 1786306680304237 us; error 51 us; skew 500 ppm
I20260809 20:18:00.304612 13215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:18:00 dist-test-slave-6k22 krb5kdc[12503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306680, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:18:00.306806 13215 init.cc:377] Logged in from keytab as oryx/127.5.61.60@KRBTEST.COM (short username oryx)
I20260809 20:18:00.308413 13215 webserver.cc:460] Webserver started at http://127.5.61.60:40421/ using document root <none> and password file <none>
I20260809 20:18:00.308714 13215 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:00.308833 13215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:00.311470 13215 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 20:18:00.313552 13452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:00.314035 13215 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260809 20:18:00.314177 13215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/wal
uuid: "32de6f510b03403ead9eca45eae81f33"
format_stamp: "Formatted at 2026-08-09 20:17:59 on dist-test-slave-6k22"
server_key: "3ad8f6614772f1394346897daef93e58"
server_key_iv: "5e99e60a0a2fa261231517989c222af4"
server_key_version: "encryptionkey@0"
I20260809 20:18:00.314370 13215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:00.325680 13215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:00.329108 13215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:00.356551 13215 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.60:44599
I20260809 20:18:00.356596 13507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.60:44599 every 8 connection(s)
I20260809 20:18:00.357486 13215 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/master-2/data/info.pb
I20260809 20:18:00.364732 13508 sys_catalog.cc:263] Verifying existing consensus state
I20260809 20:18:00.368903 13508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33: Bootstrap starting.
I20260809 20:18:00.383088 13508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 20:18:00.383721 13508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33: Bootstrap complete.
I20260809 20:18:00.384618 13508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: NON_VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: true } }
I20260809 20:18:00.384881 13508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 32de6f510b03403ead9eca45eae81f33, State: Initialized, Role: LEARNER
I20260809 20:18:00.385237 13508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: NON_VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: true } }
I20260809 20:18:00.388087 13511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: NON_VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: true } } }
I20260809 20:18:00.388408 13508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:18:00.388475 13511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [sys.catalog]: This master's current role is: LEARNER
I20260809 20:18:00.409101 13522 catalog_manager.cc:1259] Loaded cluster ID: 19ac0cc23316463f8eba2700c4b6ac78
I20260809 20:18:00.409224 13522 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33: loading cluster ID for follower catalog manager: success
I20260809 20:18:00.413115 13522 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33: acquiring CA information for follower catalog manager: success
I20260809 20:18:00.415477 13522 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 20:18:00.592137 13327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:00.584232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:44599 (local address 127.0.0.1:53000)
0809 20:18:00.584475 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:00.584500 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:00.584772 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:00.585803 (+ 1031us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:00.585807 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:00.585835 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:00.586017 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:00.586025 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.588056 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.588060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:00.589610 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:00.589640 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.591454 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.591459 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:00.591657 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:00.591698 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 20:18:00.591787 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 20:18:00.591895 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 20:18:00.592386 13523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:00.584448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53000 (local address 127.5.61.60:44599)
0809 20:18:00.585389 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:00.585397 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:00.585425 (+ 28us) server_negotiation.cc:407] Connection header received
0809 20:18:00.585505 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:00.585510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:00.585613 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:00.585746 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:00.586187 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.587863 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.589806 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.591221 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.591358 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:00.591626 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:00.591839 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 20:18:00.591995 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":511,"threads_started":1}
I20260809 20:18:00.593614 13481 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 20:18:00.594426 13511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6798f13fda9446ddae84977ff493a57e. Latest consensus state: current_term: 1 leader_uuid: "6798f13fda9446ddae84977ff493a57e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: NON_VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: true } } }
I20260809 20:18:00.594777 13511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [sys.catalog]: This master's current role is: LEARNER
W20260809 20:18:00.679742 13450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:00.680089 13450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:00.680256 13450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:00.687397 13450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:00.687551 13450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:00.687777 13450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:18:00.697959 13450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.1.keytab
--principal=oryx/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:37061
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=39695
--tserver_master_addrs=127.5.61.62:40647,127.5.61.61:42207,127.5.61.60:44599
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:00.699273 13450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:00.700609 13450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:00.713882 13529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:00.713914 13532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:00.713954 13530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:00.715098 13450 server_base.cc:1034] running on GCE node
I20260809 20:18:00.715745 13450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:00.716703 13450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:00.717916 13450 hybrid_clock.cc:648] HybridClock initialized: now 1786306680717875 us; error 57 us; skew 500 ppm
I20260809 20:18:00.718170 13450 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:18:00 dist-test-slave-6k22 krb5kdc[12503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306680, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:18:00.721452 13450 init.cc:377] Logged in from keytab as oryx/127.5.61.1@KRBTEST.COM (short username oryx)
I20260809 20:18:00.722837 13450 webserver.cc:460] Webserver started at http://127.5.61.1:39695/ using document root <none> and password file <none>
I20260809 20:18:00.723598 13450 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:00.723771 13450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:00.729241 13450 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:18:00.732028 13539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:00.733604 13450 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:18:00.733777 13450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "5b8af4b6cf364736a928ce119438e9a3"
format_stamp: "Formatted at 2026-08-09 20:17:57 on dist-test-slave-6k22"
server_key: "2f09bb317b7c221e65440c8c0b08f1e4"
server_key_iv: "ba21d555a17410a81b4a0b079fca3238"
server_key_version: "encryptionkey@0"
I20260809 20:18:00.735023 13450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:00.751365 13450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:00.755074 13450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:00.755301 13450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:00.756196 13450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:18:00.757445 13450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:18:00.757530 13450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:00.757607 13450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:18:00.757671 13450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:00.792079 13450 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:37061
I20260809 20:18:00.792119 13666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:37061 every 8 connection(s)
I20260809 20:18:00.793497 13450 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:18:00.802470 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 13450
I20260809 20:18:00.802802 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 12785
Aug 09 20:18:00 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306680, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM
Aug 09 20:18:00 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.61.2@KRBTEST.COM
Aug 09 20:18:00 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306680, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.60@KRBTEST.COM
I20260809 20:18:00.838805 13672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:00.796749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:59483 (local address 127.5.61.62:40647)
0809 20:18:00.817063 (+ 20314us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:00.817072 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:00.817100 (+ 28us) server_negotiation.cc:407] Connection header received
0809 20:18:00.817189 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:00.817198 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:00.817295 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:00.820393 (+ 3098us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:00.820648 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.822078 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.825354 (+ 3276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.825798 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.825948 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:00.830593 (+ 4645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:00.830601 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:00.830607 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:00.830665 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:00.834207 (+ 3542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:00.835215 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:00.835222 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:00.835226 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:00.835372 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:00.835990 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:00.835996 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:00.835999 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:00.836381 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:00.836524 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:00.838133 (+ 1609us) server_negotiation.cc:299] Negotiation successful
0809 20:18:00.838407 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20145,"thread_start_us":20687,"threads_started":1}
Aug 09 20:18:00 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306680, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.61@KRBTEST.COM
I20260809 20:18:00.839195 13679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:00.804075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34564 (local address 127.5.61.2:38017)
0809 20:18:00.813387 (+ 9312us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:00.813399 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:00.814580 (+ 1181us) server_negotiation.cc:407] Connection header received
0809 20:18:00.814673 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:00.814685 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:00.814975 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:00.815176 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:00.817921 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.819918 (+ 1997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.822333 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.823534 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.823799 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:00.831026 (+ 7227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:00.831040 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:00.831055 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:00.831105 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:00.834892 (+ 3787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:00.835761 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:00.835769 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:00.835773 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:00.835943 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:00.836430 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:00.836436 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:00.836441 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:00.837253 (+ 812us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:00.837522 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:00.838506 (+ 984us) server_negotiation.cc:299] Negotiation successful
0809 20:18:00.838760 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9080,"thread_start_us":9310,"threads_started":1}
I20260809 20:18:00.840698 13673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:00.797284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.5.61.1:59483)
0809 20:18:00.798206 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:00.798258 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:00.801345 (+ 3087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:00.817969 (+ 16624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:00.817987 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:00.819082 (+ 1095us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:00.820145 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:00.820187 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.822366 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.822374 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:00.825213 (+ 2839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:00.825230 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.826047 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.826055 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:00.826293 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:00.826927 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:00.826955 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:00.830258 (+ 3303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:00.834399 (+ 4141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:00.834408 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:00.834425 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:00.835067 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:00.835514 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:00.835520 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:00.835523 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:00.835845 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:00.836603 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:00.836610 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:00.837071 (+ 461us) client_negotiation.cc:769] Sending connection context
0809 20:18:00.837434 (+ 363us) client_negotiation.cc:240] Negotiation successful
0809 20:18:00.839187 (+ 1753us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"spinlock_wait_cycles":21120,"thread_start_us":496,"threads_started":1}
I20260809 20:18:00.841183 12930 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:00.842918 13668 heartbeater.cc:347] Connected to a master server at 127.5.61.62:40647
I20260809 20:18:00.843338 13668 heartbeater.cc:464] Registering TS with master...
I20260809 20:18:00.844465 13668 heartbeater.cc:511] Master 127.5.61.62:40647 requested a full tablet report, sending...
I20260809 20:18:00.846499 12559 ts_manager.cc:194] Re-registered known tserver with Master: 5b8af4b6cf364736a928ce119438e9a3 (127.5.61.1:37061)
I20260809 20:18:00.847709 12559 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.61.1@KRBTEST.COM'} at 127.5.61.1:59483
I20260809 20:18:00.849766 13674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:00.798880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:42207 (local address 127.5.61.1:43961)
0809 20:18:00.829176 (+ 30296us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:00.829200 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:00.829314 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:00.829941 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:00.829945 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:00.830634 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:00.830958 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:00.830970 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.833591 (+ 2621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.833596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:00.835057 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:00.835070 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.836364 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.836371 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:00.836577 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:00.837187 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:00.837209 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:00.840024 (+ 2815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:00.846670 (+ 6646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:00.846675 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:00.846678 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:00.847157 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:00.847702 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:00.847709 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:00.847715 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:00.847911 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:00.848791 (+ 880us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:00.848799 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:00.848991 (+ 192us) client_negotiation.cc:769] Sending connection context
0809 20:18:00.849169 (+ 178us) client_negotiation.cc:240] Negotiation successful
0809 20:18:00.849476 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30157,"thread_start_us":29318,"threads_started":1}
I20260809 20:18:00.850660 13676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:00.799260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:43961 (local address 127.5.61.61:42207)
0809 20:18:00.828008 (+ 28748us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:00.828015 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:00.829422 (+ 1407us) server_negotiation.cc:407] Connection header received
0809 20:18:00.829495 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:00.829500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:00.829588 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:00.829712 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:00.831119 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.833391 (+ 2272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.835230 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.835939 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.836045 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:00.840239 (+ 4194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:00.840245 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:00.840250 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:00.840301 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:00.846456 (+ 6155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:00.847306 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:00.847315 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:00.847321 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:00.847492 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:00.848056 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:00.848064 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:00.848069 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:00.848499 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:00.848680 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:00.850071 (+ 1391us) server_negotiation.cc:299] Negotiation successful
0809 20:18:00.850267 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28599,"thread_start_us":28479,"threads_started":1}
I20260809 20:18:00.851600 13669 heartbeater.cc:347] Connected to a master server at 127.5.61.61:42207
I20260809 20:18:00.851733 13669 heartbeater.cc:464] Registering TS with master...
I20260809 20:18:00.851958 13523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:00.797071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:55971 (local address 127.5.61.60:44599)
0809 20:18:00.797325 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:00.797335 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:00.829028 (+ 31693us) server_negotiation.cc:407] Connection header received
0809 20:18:00.829105 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:00.829117 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:00.829238 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:00.829345 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:00.830817 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.831986 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.833852 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.834483 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.834571 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:00.839747 (+ 5176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:00.839754 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:00.839759 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:00.839798 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:00.843265 (+ 3467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:00.844419 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:00.844425 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:00.844428 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:00.844661 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:00.845169 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:00.845175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:00.845179 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:00.845541 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:00.845669 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:00.851310 (+ 5641us) server_negotiation.cc:299] Negotiation successful
0809 20:18:00.851614 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 20:18:00.852159 13669 heartbeater.cc:511] Master 127.5.61.61:42207 requested a full tablet report, sending...
I20260809 20:18:00.852833 13675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:00.797284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:44599 (local address 127.5.61.1:55971)
0809 20:18:00.828490 (+ 31206us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:00.828509 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:00.828638 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:00.829527 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:00.829531 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:00.830300 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:00.830623 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:00.830633 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.832181 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.832188 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:00.833678 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:00.833697 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:00.834826 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:00.834831 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:00.835009 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:00.835610 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:00.835634 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:00.839458 (+ 3824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:00.843387 (+ 3929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:00.843447 (+ 60us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:00.843450 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:00.844268 (+ 818us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:00.844832 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:00.844839 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:00.844845 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:00.845024 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:00.845751 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:00.845755 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:00.845895 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 20:18:00.852150 (+ 6255us) client_negotiation.cc:240] Negotiation successful
0809 20:18:00.852449 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31053,"mutex_wait_us":316,"thread_start_us":31263,"threads_started":1}
I20260809 20:18:00.853724 13667 heartbeater.cc:347] Connected to a master server at 127.5.61.60:44599
I20260809 20:18:00.853868 13667 heartbeater.cc:464] Registering TS with master...
I20260809 20:18:00.853989 13257 ts_manager.cc:194] Registered new tserver with Master: 5b8af4b6cf364736a928ce119438e9a3 (127.5.61.1:37061)
I20260809 20:18:00.854206 13667 heartbeater.cc:511] Master 127.5.61.60:44599 requested a full tablet report, sending...
I20260809 20:18:00.856021 13469 ts_manager.cc:194] Registered new tserver with Master: 5b8af4b6cf364736a928ce119438e9a3 (127.5.61.1:37061)
I20260809 20:18:00.930944 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 12785
I20260809 20:18:00.952118 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:38017
--local_ip_for_outbound_sockets=127.5.61.2
--tserver_master_addrs=127.5.61.62:40647
--webserver_port=39987
--webserver_interface=127.5.61.2
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.2.keytab
--principal=oryx/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.61.62:40647,127.5.61.61:42207
--tserver_master_addrs=127.5.61.62:40647,127.5.61.61:42207,127.5.61.60:44599 with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 20:18:01.110029 13423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6798f13fda9446ddae84977ff493a57e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: NON_VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: true } } }
I20260809 20:18:01.110399 13423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [sys.catalog]: This master's current role is: LEADER
I20260809 20:18:01.111924 13423 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e: attempting to promote NON_VOTER 32de6f510b03403ead9eca45eae81f33 to VOTER
I20260809 20:18:01.112756 13423 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: false } }
I20260809 20:18:01.114949 13481 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [term 1 LEARNER]: Refusing update from remote peer 6798f13fda9446ddae84977ff493a57e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 20:18:01.115689 13423 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [LEADER]: Connected to new peer: Peer: permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 20:18:01.116031 13268 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 1 FOLLOWER]: Refusing update from remote peer 6798f13fda9446ddae84977ff493a57e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 20:18:01.116796 13423 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 20:18:01.119583 13683 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 32de6f510b03403ead9eca45eae81f33 (127.5.61.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: false } } }
I20260809 20:18:01.120268 13268 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 32de6f510b03403ead9eca45eae81f33 (127.5.61.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: false } } }
I20260809 20:18:01.122404 13683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6798f13fda9446ddae84977ff493a57e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: false } } }
I20260809 20:18:01.122659 13683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [sys.catalog]: This master's current role is: LEADER
I20260809 20:18:01.123248 13481 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 32de6f510b03403ead9eca45eae81f33 (127.5.61.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: false } } }
I20260809 20:18:01.124084 13423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6798f13fda9446ddae84977ff493a57e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: false } } }
I20260809 20:18:01.124361 13423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [sys.catalog]: This master's current role is: LEADER
I20260809 20:18:01.124701 13683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6798f13fda9446ddae84977ff493a57e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: false } } }
I20260809 20:18:01.125029 13683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e [sys.catalog]: This master's current role is: LEADER
I20260809 20:18:01.124929 13686 sys_catalog.cc:455] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6798f13fda9446ddae84977ff493a57e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: false } } }
I20260809 20:18:01.125401 13686 sys_catalog.cc:458] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:18:01.128095 13688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6798f13fda9446ddae84977ff493a57e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: false } } }
I20260809 20:18:01.128352 13688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 20:18:01.294178 13681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:01.294499 13681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:01.294672 13681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:01.302183 13681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:01.302307 13681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:01.302421 13681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:18:01.312269 13681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.2.keytab
--principal=oryx/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:38017
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=39987
--tserver_master_addrs=127.5.61.62:40647,127.5.61.61:42207,127.5.61.60:44599
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:01.313476 13681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:01.314845 13681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:01.328212 13693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:01.328181 13694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:01.328529 13696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:01.329353 13681 server_base.cc:1034] running on GCE node
I20260809 20:18:01.330154 13681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:01.331032 13681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:01.332294 13681 hybrid_clock.cc:648] HybridClock initialized: now 1786306681332235 us; error 64 us; skew 500 ppm
I20260809 20:18:01.332657 13681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:18:01 dist-test-slave-6k22 krb5kdc[12503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306681, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:18:01.335944 13681 init.cc:377] Logged in from keytab as oryx/127.5.61.2@KRBTEST.COM (short username oryx)
I20260809 20:18:01.337335 13681 webserver.cc:460] Webserver started at http://127.5.61.2:39987/ using document root <none> and password file <none>
I20260809 20:18:01.338061 13681 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:01.338227 13681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:01.343852 13681 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 20:18:01.346575 13704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:01.347982 13681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:18:01.348104 13681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "07f1c4c3d4ee43b281427f86818468c1"
format_stamp: "Formatted at 2026-08-09 20:17:58 on dist-test-slave-6k22"
server_key: "367c5c904edf7143d7a55e0e402758dc"
server_key_iv: "d9887f5a33c732d9ce6495c5e4328523"
server_key_version: "encryptionkey@0"
I20260809 20:18:01.348958 13681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:01.379817 13681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:01.383700 13681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:01.383937 13681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:01.384817 13681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:18:01.386112 13681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:18:01.386199 13681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:01.386274 13681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:18:01.386329 13681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:01.420441 13681 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:38017
I20260809 20:18:01.420549 13833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:38017 every 8 connection(s)
I20260809 20:18:01.421877 13681 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:18:01.425463 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 13681
I20260809 20:18:01.425824 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 12941
Aug 09 20:18:01 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.61.3@KRBTEST.COM
Aug 09 20:18:01 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306681, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM
Aug 09 20:18:01 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306681, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.60@KRBTEST.COM
I20260809 20:18:01.474049 13844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:01.427020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42422 (local address 127.5.61.3:33467)
0809 20:18:01.442264 (+ 15244us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:01.442277 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:01.444604 (+ 2327us) server_negotiation.cc:407] Connection header received
0809 20:18:01.444757 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:01.444772 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:01.445345 (+ 573us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:01.445580 (+ 235us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:01.449785 (+ 4205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:01.452055 (+ 2270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:01.454567 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:01.456047 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:01.456290 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:01.464667 (+ 8377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:01.464683 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:01.464699 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:01.464745 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:01.469121 (+ 4376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:01.470790 (+ 1669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:01.470800 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:01.470803 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:01.470983 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:01.471499 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:01.471506 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:01.471509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:01.471969 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:01.472762 (+ 793us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:01.473290 (+ 528us) server_negotiation.cc:299] Negotiation successful
0809 20:18:01.473562 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14985,"thread_start_us":15227,"threads_started":1}
I20260809 20:18:01.475468 13084 generic_service.cc:196] Checking for leaks (request via RPC)
Aug 09 20:18:01 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306681, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.61@KRBTEST.COM
I20260809 20:18:01.482770 13839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:01.425218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:42147 (local address 127.5.61.62:40647)
0809 20:18:01.425711 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:01.425718 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:01.453333 (+ 27615us) server_negotiation.cc:407] Connection header received
0809 20:18:01.457105 (+ 3772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:01.457112 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:01.457215 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:01.457333 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:01.461815 (+ 4482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:01.462987 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:01.466296 (+ 3309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:01.467119 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:01.467228 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:01.472161 (+ 4933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:01.472167 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:01.472171 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:01.472223 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:01.477171 (+ 4948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:01.479712 (+ 2541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:01.479719 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:01.479723 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:01.480009 (+ 286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:01.480522 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:01.480529 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:01.480533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:01.480918 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:01.481113 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:01.482007 (+ 894us) server_negotiation.cc:299] Negotiation successful
0809 20:18:01.482258 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":983,"threads_started":1}
I20260809 20:18:01.482801 13841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:01.426123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:41269 (local address 127.5.61.60:44599)
0809 20:18:01.449940 (+ 23817us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:01.449952 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:01.457664 (+ 7712us) server_negotiation.cc:407] Connection header received
0809 20:18:01.457942 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:01.457947 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:01.458035 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:01.458141 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:01.462296 (+ 4155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:01.463469 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:01.466290 (+ 2821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:01.467112 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:01.467254 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:01.473707 (+ 6453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:01.473716 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:01.473722 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:01.473792 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:01.478143 (+ 4351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:01.479544 (+ 1401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:01.479552 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:01.479584 (+ 32us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:01.479768 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:01.480488 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:01.480495 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:01.480498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:01.480917 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:01.481102 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:01.482009 (+ 907us) server_negotiation.cc:299] Negotiation successful
0809 20:18:01.482258 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23567,"thread_start_us":23400,"threads_started":1}
I20260809 20:18:01.484699 13840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:01.425786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.5.61.2:42147)
0809 20:18:01.453115 (+ 27329us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:01.453193 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:01.456736 (+ 3543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:01.458217 (+ 1481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:01.458231 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:01.461248 (+ 3017us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:01.461598 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:01.461614 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:01.463224 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:01.463229 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:01.466095 (+ 2866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:01.466114 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:01.467654 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:01.467663 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:01.468090 (+ 427us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:01.468877 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:01.468916 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:01.471867 (+ 2951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:01.478883 (+ 7016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:01.478891 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:01.478898 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:01.479576 (+ 678us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:01.480191 (+ 615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:01.480196 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:01.480200 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:01.480388 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:01.481090 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:01.481098 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:01.481538 (+ 440us) client_negotiation.cc:769] Sending connection context
0809 20:18:01.481856 (+ 318us) client_negotiation.cc:240] Negotiation successful
0809 20:18:01.482477 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26902,"mutex_wait_us":163,"thread_start_us":26555,"threads_started":1}
I20260809 20:18:01.484745 13842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:01.425785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:44599 (local address 127.5.61.2:41269)
0809 20:18:01.457554 (+ 31769us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:01.457575 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:01.457788 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:01.458360 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:01.458364 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:01.459409 (+ 1045us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:01.460541 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:01.460573 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:01.463956 (+ 3383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:01.463965 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:01.466095 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:01.466113 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:01.467654 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:01.467663 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:01.468513 (+ 850us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:01.470264 (+ 1751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:01.470299 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:01.473422 (+ 3123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:01.478347 (+ 4925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:01.478362 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:01.478403 (+ 41us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:01.479345 (+ 942us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:01.479995 (+ 650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:01.480001 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:01.480004 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:01.480282 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:01.481234 (+ 952us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:01.481241 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:01.481539 (+ 298us) client_negotiation.cc:769] Sending connection context
0809 20:18:01.483082 (+ 1543us) client_negotiation.cc:240] Negotiation successful
0809 20:18:01.483452 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31603,"mutex_wait_us":865,"spinlock_wait_cycles":55296,"thread_start_us":31471,"threads_started":1}
I20260809 20:18:01.486852 13836 heartbeater.cc:347] Connected to a master server at 127.5.61.62:40647
I20260809 20:18:01.486879 13834 heartbeater.cc:347] Connected to a master server at 127.5.61.60:44599
I20260809 20:18:01.487221 13836 heartbeater.cc:464] Registering TS with master...
I20260809 20:18:01.487231 13834 heartbeater.cc:464] Registering TS with master...
I20260809 20:18:01.488363 13834 heartbeater.cc:511] Master 127.5.61.60:44599 requested a full tablet report, sending...
I20260809 20:18:01.489748 13836 heartbeater.cc:511] Master 127.5.61.62:40647 requested a full tablet report, sending...
I20260809 20:18:01.490916 13469 ts_manager.cc:194] Registered new tserver with Master: 07f1c4c3d4ee43b281427f86818468c1 (127.5.61.2:38017)
I20260809 20:18:01.490965 12559 ts_manager.cc:194] Re-registered known tserver with Master: 07f1c4c3d4ee43b281427f86818468c1 (127.5.61.2:38017)
I20260809 20:18:01.491747 13846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:01.458802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:42207 (local address 127.5.61.2:45603)
0809 20:18:01.462267 (+ 3465us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:01.462289 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:01.462424 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:01.462923 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:01.462928 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:01.463597 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:01.464325 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:01.464339 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:01.466049 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:01.466058 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:01.468792 (+ 2734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:01.468822 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:01.470748 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:01.470764 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:01.471040 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:01.471853 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:01.471888 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:01.481651 (+ 9763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:01.488329 (+ 6678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:01.488337 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:01.488349 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:01.488929 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:01.489474 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:01.489483 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:01.489490 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:01.489716 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:01.490514 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:01.490520 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:01.490704 (+ 184us) client_negotiation.cc:769] Sending connection context
0809 20:18:01.490854 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 20:18:01.491181 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3341,"mutex_wait_us":740,"spinlock_wait_cycles":3968,"thread_start_us":3211,"threads_started":1}
I20260809 20:18:01.492345 12559 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.61.2@KRBTEST.COM'} at 127.5.61.2:42147
I20260809 20:18:01.492643 13845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:01.454020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:45603 (local address 127.5.61.61:42207)
0809 20:18:01.461997 (+ 7977us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:01.462003 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:01.462545 (+ 542us) server_negotiation.cc:407] Connection header received
0809 20:18:01.462635 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:01.462657 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:01.462746 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:01.463715 (+ 969us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:01.464524 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:01.465848 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:01.469380 (+ 3532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:01.470218 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:01.470373 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:01.483145 (+ 12772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:01.483166 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:01.483172 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:01.483240 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:01.488076 (+ 4836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:01.489125 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:01.489134 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:01.489140 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:01.489308 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:01.489880 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:01.489885 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:01.489889 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:01.490286 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:01.490445 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:01.492059 (+ 1614us) server_negotiation.cc:299] Negotiation successful
0809 20:18:01.492300 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7840,"thread_start_us":7720,"threads_started":1}
I20260809 20:18:01.495415 13837 heartbeater.cc:347] Connected to a master server at 127.5.61.61:42207
I20260809 20:18:01.495548 13837 heartbeater.cc:464] Registering TS with master...
I20260809 20:18:01.496049 13837 heartbeater.cc:511] Master 127.5.61.61:42207 requested a full tablet report, sending...
I20260809 20:18:01.497270 13257 ts_manager.cc:194] Registered new tserver with Master: 07f1c4c3d4ee43b281427f86818468c1 (127.5.61.2:38017)
I20260809 20:18:01.564081 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 12941
I20260809 20:18:01.588220 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:33467
--local_ip_for_outbound_sockets=127.5.61.3
--tserver_master_addrs=127.5.61.62:40647
--webserver_port=34287
--webserver_interface=127.5.61.3
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.3.keytab
--principal=oryx/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.61.62:40647,127.5.61.61:42207
--tserver_master_addrs=127.5.61.62:40647,127.5.61.61:42207,127.5.61.60:44599 with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 20:18:01.850965 13668 heartbeater.cc:503] Master 127.5.61.62:40647 was elected leader, sending a full tablet report...
W20260809 20:18:01.926617 13849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:01.926934 13849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:01.927107 13849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:01.934643 13849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:01.934780 13849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:01.934958 13849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:18:01.944690 13849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.3.keytab
--principal=oryx/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:33467
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=34287
--tserver_master_addrs=127.5.61.62:40647,127.5.61.61:42207,127.5.61.60:44599
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:01.945958 13849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:01.947300 13849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:01.960665 13854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:01.960847 13857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:01.960664 13855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:01.962118 13849 server_base.cc:1034] running on GCE node
I20260809 20:18:01.962705 13849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:01.963694 13849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:01.964915 13849 hybrid_clock.cc:648] HybridClock initialized: now 1786306681964858 us; error 53 us; skew 500 ppm
I20260809 20:18:01.965232 13849 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:18:01 dist-test-slave-6k22 krb5kdc[12503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306681, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:18:01.968405 13849 init.cc:377] Logged in from keytab as oryx/127.5.61.3@KRBTEST.COM (short username oryx)
I20260809 20:18:01.969946 13849 webserver.cc:460] Webserver started at http://127.5.61.3:34287/ using document root <none> and password file <none>
I20260809 20:18:01.970654 13849 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:01.970827 13849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:01.976109 13849 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 20:18:01.978832 13865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:01.980298 13849 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:18:01.980492 13849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "408146e0c09b428b8f9dd510aeecbdc6"
format_stamp: "Formatted at 2026-08-09 20:17:58 on dist-test-slave-6k22"
server_key: "d7dac5141b288c25652067abc2c5acc2"
server_key_iv: "1a0da5fa3c9f2e6cbf61155169b3d56f"
server_key_version: "encryptionkey@0"
I20260809 20:18:01.981472 13849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:01.999329 13849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:02.003629 13849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:02.003917 13849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:02.004982 13849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:18:02.006462 13849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:18:02.006587 13849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:02.006721 13849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:18:02.006812 13849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:02.042083 13849 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:33467
I20260809 20:18:02.042132 13996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:33467 every 8 connection(s)
I20260809 20:18:02.043793 13849 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:18:02.051679 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 13849
Aug 09 20:18:02 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306681, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.60@KRBTEST.COM
Aug 09 20:18:02 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306681, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM
I20260809 20:18:02.109292 14003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:02.047896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:43177 (local address 127.5.61.60:44599)
0809 20:18:02.075956 (+ 28060us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:02.075960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:02.082600 (+ 6640us) server_negotiation.cc:407] Connection header received
0809 20:18:02.082891 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:02.082898 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:02.082990 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:02.083073 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:02.087721 (+ 4648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.089440 (+ 1719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.091388 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.092288 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.092436 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:02.100788 (+ 8352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:02.100798 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:02.100806 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:02.100871 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:02.105633 (+ 4762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:02.106491 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:02.106499 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:02.106504 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:02.106708 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:02.107204 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:02.107210 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:02.107213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:02.107641 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:02.107791 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:02.108673 (+ 882us) server_negotiation.cc:299] Negotiation successful
0809 20:18:02.108892 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27925,"thread_start_us":27802,"threads_started":1}
I20260809 20:18:02.111099 14000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:02.046729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:48577 (local address 127.5.61.62:40647)
0809 20:18:02.063157 (+ 16428us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:02.063168 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:02.076012 (+ 12844us) server_negotiation.cc:407] Connection header received
0809 20:18:02.078366 (+ 2354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:02.078393 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:02.078514 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:02.078637 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:02.084873 (+ 6236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.085824 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.088319 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.089069 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.089218 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:02.099612 (+ 10394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:02.099630 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:02.099638 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:02.099691 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:02.104613 (+ 4922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:02.105786 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:02.105792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:02.105796 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:02.105975 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:02.106672 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:02.106678 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:02.106681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:02.107065 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:02.107259 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:02.109857 (+ 2598us) server_negotiation.cc:299] Negotiation successful
0809 20:18:02.110101 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16228,"thread_start_us":16438,"threads_started":1}
I20260809 20:18:02.113619 14002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:02.047468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:44599 (local address 127.5.61.3:43177)
0809 20:18:02.082478 (+ 35010us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:02.082499 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:02.082733 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:02.083334 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:02.083339 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:02.084288 (+ 949us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:02.084575 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:02.084588 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.089633 (+ 5045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.089639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:02.091188 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:02.091208 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.092741 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.092749 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:02.093044 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:02.093924 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:02.093964 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:02.098267 (+ 4303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:02.105806 (+ 7539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:02.105811 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:02.105814 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:02.106330 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:02.106860 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:02.106866 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:02.106869 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:02.107037 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:02.107857 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:02.107864 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:02.108011 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 20:18:02.108324 (+ 313us) client_negotiation.cc:240] Negotiation successful
0809 20:18:02.111903 (+ 3579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34841,"mutex_wait_us":3796,"thread_start_us":15612,"threads_started":1}
I20260809 20:18:02.114481 14004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:02.047467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.5.61.3:48577)
0809 20:18:02.064289 (+ 16822us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:02.064344 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:02.078005 (+ 13661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:02.079923 (+ 1918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:02.079936 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:02.080843 (+ 907us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:02.081723 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:02.081749 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.086102 (+ 4353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.086105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:02.087466 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:02.087479 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.089519 (+ 2040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.089529 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:02.093499 (+ 3970us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:02.095723 (+ 2224us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:02.095756 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:02.099311 (+ 3555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:02.104836 (+ 5525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:02.104848 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:02.104867 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:02.105611 (+ 744us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:02.106161 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:02.106168 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:02.106175 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:02.106522 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:02.107386 (+ 864us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:02.107395 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:02.107964 (+ 569us) client_negotiation.cc:769] Sending connection context
0809 20:18:02.108324 (+ 360us) client_negotiation.cc:240] Negotiation successful
0809 20:18:02.108901 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16485,"mutex_wait_us":4,"spinlock_wait_cycles":19200,"thread_start_us":34507,"threads_started":1}
I20260809 20:18:02.116607 13997 heartbeater.cc:347] Connected to a master server at 127.5.61.60:44599
I20260809 20:18:02.117663 13997 heartbeater.cc:464] Registering TS with master...
I20260809 20:18:02.118731 13997 heartbeater.cc:511] Master 127.5.61.60:44599 requested a full tablet report, sending...
I20260809 20:18:02.121009 13469 ts_manager.cc:194] Registered new tserver with Master: 408146e0c09b428b8f9dd510aeecbdc6 (127.5.61.3:33467)
I20260809 20:18:02.123502 13998 heartbeater.cc:347] Connected to a master server at 127.5.61.62:40647
I20260809 20:18:02.123660 13998 heartbeater.cc:464] Registering TS with master...
I20260809 20:18:02.124157 13998 heartbeater.cc:511] Master 127.5.61.62:40647 requested a full tablet report, sending...
I20260809 20:18:02.124882 14016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:02.090762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53012 (local address 127.5.61.60:44599)
0809 20:18:02.098149 (+ 7387us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:02.098157 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:02.100409 (+ 2252us) server_negotiation.cc:407] Connection header received
0809 20:18:02.100475 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:02.100483 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:02.100572 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:02.100669 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:02.105418 (+ 4749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.107222 (+ 1804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.110074 (+ 2852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.111044 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.111188 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:02.115417 (+ 4229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:02.115426 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:02.115438 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:02.115547 (+ 109us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:02.120358 (+ 4811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:02.121277 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:02.121298 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:02.121304 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:02.121480 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:02.121894 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:02.121899 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:02.121902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:02.122261 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:02.122499 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:02.123689 (+ 1190us) server_negotiation.cc:299] Negotiation successful
0809 20:18:02.123904 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7268,"thread_start_us":7233,"threads_started":1}
I20260809 20:18:02.126101 12559 ts_manager.cc:194] Re-registered known tserver with Master: 408146e0c09b428b8f9dd510aeecbdc6 (127.5.61.3:33467)
I20260809 20:18:02.127341 12559 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.61.3@KRBTEST.COM'} at 127.5.61.3:48577
Aug 09 20:18:02 dist-test-slave-6k22 krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306681, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.61@KRBTEST.COM
I20260809 20:18:02.133504 14017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:02.091065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55116 (local address 127.5.61.62:40647)
0809 20:18:02.102474 (+ 11409us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:02.102482 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:02.102500 (+ 18us) server_negotiation.cc:407] Connection header received
0809 20:18:02.102549 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:02.102553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:02.102642 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:02.102763 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:02.114057 (+ 11294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.115289 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.119437 (+ 4148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.120228 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.120336 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:02.126916 (+ 6580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:02.126924 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:02.126929 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:02.126968 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:02.130493 (+ 3525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:02.131139 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:02.131144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:02.131147 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:02.131305 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:02.131824 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:02.131831 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:02.131835 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:02.132167 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:02.132413 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:02.132856 (+ 443us) server_negotiation.cc:299] Negotiation successful
0809 20:18:02.133088 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11240,"thread_start_us":11058,"threads_started":1}
I20260809 20:18:02.137954 14012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:02.088990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:42207 (local address 127.5.61.3:57339)
0809 20:18:02.118399 (+ 29409us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:02.118431 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:02.118589 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:02.119143 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:02.119148 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:02.119869 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:02.120192 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:02.120203 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.121950 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.121955 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:02.124016 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:02.124031 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.126383 (+ 2352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.126391 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:02.126580 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:02.127201 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:02.127229 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:02.130200 (+ 2971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:02.134769 (+ 4569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:02.134774 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:02.134778 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:02.135097 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:02.135586 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:02.135594 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:02.135600 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:02.135789 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:02.136668 (+ 879us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:02.136673 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:02.136864 (+ 191us) client_negotiation.cc:769] Sending connection context
0809 20:18:02.137046 (+ 182us) client_negotiation.cc:240] Negotiation successful
0809 20:18:02.137412 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29265,"thread_start_us":23278,"threads_started":1}
I20260809 20:18:02.137984 14019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:02.082922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:57339 (local address 127.5.61.61:42207)
0809 20:18:02.098130 (+ 15208us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:02.098137 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:02.118531 (+ 20394us) server_negotiation.cc:407] Connection header received
0809 20:18:02.118714 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:02.118722 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:02.118819 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:02.118926 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:02.120389 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.121768 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.124212 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.125129 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.125275 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:02.130470 (+ 5195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:02.130477 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:02.130484 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:02.130548 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:02.134624 (+ 4076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:02.135229 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:02.135234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:02.135238 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:02.135405 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:02.136195 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:02.136198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:02.136200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:02.136533 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:02.136709 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:02.137337 (+ 628us) server_negotiation.cc:299] Negotiation successful
0809 20:18:02.137558 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15082,"thread_start_us":41877,"threads_started":1}
I20260809 20:18:02.139262 13999 heartbeater.cc:347] Connected to a master server at 127.5.61.61:42207
I20260809 20:18:02.139413 13999 heartbeater.cc:464] Registering TS with master...
I20260809 20:18:02.139899 13999 heartbeater.cc:511] Master 127.5.61.61:42207 requested a full tablet report, sending...
I20260809 20:18:02.141215 13257 ts_manager.cc:194] Registered new tserver with Master: 408146e0c09b428b8f9dd510aeecbdc6 (127.5.61.3:33467)
I20260809 20:18:02.144397 14011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:02.095451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46094 (local address 127.5.61.61:42207)
0809 20:18:02.125268 (+ 29817us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:02.125273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:02.125291 (+ 18us) server_negotiation.cc:407] Connection header received
0809 20:18:02.125334 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:02.125338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:02.125407 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:02.125509 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:02.128860 (+ 3351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.130018 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.132013 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.132737 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.132870 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:02.136653 (+ 3783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:02.136661 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:02.136666 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:02.136722 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:02.141593 (+ 4871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:02.142332 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:02.142337 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:02.142341 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:02.142479 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:02.142882 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:02.142888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:02.142892 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:02.143159 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:02.143418 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:02.143831 (+ 413us) server_negotiation.cc:299] Negotiation successful
0809 20:18:02.144034 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29692,"thread_start_us":1688,"threads_started":1}
I20260809 20:18:02.137909 12559 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:18:02.146029 12559 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:18:02.183197 14034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:02.167431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:33467 (local address 127.0.0.1:42430)
0809 20:18:02.168473 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:02.168485 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:02.168563 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:02.172180 (+ 3617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:02.172185 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:02.172206 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:02.172546 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:02.172558 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.174961 (+ 2403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.174966 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:02.177327 (+ 2361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:02.177341 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.182374 (+ 5033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.182380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:02.182559 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:02.182599 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 20:18:02.182723 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 20:18:02.182845 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":940,"thread_start_us":4354,"threads_started":1}
I20260809 20:18:02.183318 14035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:02.168094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42430 (local address 127.5.61.3:33467)
0809 20:18:02.171097 (+ 3003us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:02.171106 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:02.171158 (+ 52us) server_negotiation.cc:407] Connection header received
0809 20:18:02.171264 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:02.171273 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:02.171620 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:02.171862 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:02.172777 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.174774 (+ 1997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.178443 (+ 3669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.180070 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.180173 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:02.180382 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:02.182810 (+ 2428us) server_negotiation.cc:299] Negotiation successful
0809 20:18:02.182961 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2850,"thread_start_us":2684,"threads_started":1}
I20260809 20:18:02.187418 13921 tablet_service.cc:1467] Processing CreateTablet for tablet e0d51271f2c241f9b1816003bca29381 (DEFAULT_TABLE table=test-table [id=090945c5b9704804a0a8020e4fef84cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:18:02.189062 13921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0d51271f2c241f9b1816003bca29381. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:02.190107 14031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:02.166890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:38017 (local address 127.0.0.1:34576)
0809 20:18:02.167451 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:02.167471 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:02.167619 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:02.181379 (+ 13760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:02.181383 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:02.181408 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:02.181779 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:02.181794 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.185155 (+ 3361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.185160 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:02.187228 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:02.187239 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.189183 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.189190 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:02.189433 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:02.189481 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 20:18:02.189576 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 20:18:02.189709 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"mutex_wait_us":24,"thread_start_us":289,"threads_started":1}
I20260809 20:18:02.190825 14032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:02.167030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34576 (local address 127.5.61.2:38017)
0809 20:18:02.180780 (+ 13750us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:02.180791 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:02.180823 (+ 32us) server_negotiation.cc:407] Connection header received
0809 20:18:02.180925 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:02.180933 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:02.181191 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:02.181379 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:02.183527 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.185022 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.187383 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.188965 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.189115 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:02.189330 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:02.190359 (+ 1029us) server_negotiation.cc:299] Negotiation successful
0809 20:18:02.190527 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13575,"thread_start_us":13473,"threads_started":1}
I20260809 20:18:02.192313 14033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:02.167686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37061 (local address 127.0.0.1:39360)
0809 20:18:02.172043 (+ 4357us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:02.172061 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:02.172180 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:02.184661 (+ 12481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:02.184666 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:02.184692 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:02.185123 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:02.185140 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.187050 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.187055 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:02.189384 (+ 2329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:02.189398 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.191434 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.191441 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:02.191669 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:02.191702 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 20:18:02.191798 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 20:18:02.191916 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4229,"mutex_wait_us":23,"thread_start_us":576,"threads_started":1}
I20260809 20:18:02.192317 14036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:02.168197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39360 (local address 127.5.61.1:37061)
0809 20:18:02.184039 (+ 15842us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:02.184048 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:02.184076 (+ 28us) server_negotiation.cc:407] Connection header received
0809 20:18:02.184173 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:02.184181 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:02.184404 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:02.184630 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:02.185342 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.186903 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.189601 (+ 2698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.191218 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.191349 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:02.191543 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:02.191859 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 20:18:02.192026 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15672,"thread_start_us":15563,"threads_started":1}
I20260809 20:18:02.194355 13758 tablet_service.cc:1467] Processing CreateTablet for tablet e0d51271f2c241f9b1816003bca29381 (DEFAULT_TABLE table=test-table [id=090945c5b9704804a0a8020e4fef84cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:18:02.195466 13758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0d51271f2c241f9b1816003bca29381. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:02.196069 13592 tablet_service.cc:1467] Processing CreateTablet for tablet e0d51271f2c241f9b1816003bca29381 (DEFAULT_TABLE table=test-table [id=090945c5b9704804a0a8020e4fef84cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:18:02.197320 13592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0d51271f2c241f9b1816003bca29381. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:02.204507 14037 tablet_bootstrap.cc:492] T e0d51271f2c241f9b1816003bca29381 P 408146e0c09b428b8f9dd510aeecbdc6: Bootstrap starting.
I20260809 20:18:02.207942 14038 tablet_bootstrap.cc:492] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1: Bootstrap starting.
I20260809 20:18:02.208576 14037 tablet_bootstrap.cc:654] T e0d51271f2c241f9b1816003bca29381 P 408146e0c09b428b8f9dd510aeecbdc6: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:02.209707 14037 log.cc:824] T e0d51271f2c241f9b1816003bca29381 P 408146e0c09b428b8f9dd510aeecbdc6: Log is configured to *not* fsync() on all Append() calls
I20260809 20:18:02.211388 14038 tablet_bootstrap.cc:654] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:02.212740 14037 tablet_bootstrap.cc:492] T e0d51271f2c241f9b1816003bca29381 P 408146e0c09b428b8f9dd510aeecbdc6: No bootstrap required, opened a new log
I20260809 20:18:02.212860 14038 log.cc:824] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1: Log is configured to *not* fsync() on all Append() calls
I20260809 20:18:02.213027 14037 ts_tablet_manager.cc:1397] T e0d51271f2c241f9b1816003bca29381 P 408146e0c09b428b8f9dd510aeecbdc6: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 20:18:02.214918 14040 tablet_bootstrap.cc:492] T e0d51271f2c241f9b1816003bca29381 P 5b8af4b6cf364736a928ce119438e9a3: Bootstrap starting.
I20260809 20:18:02.217365 14038 tablet_bootstrap.cc:492] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1: No bootstrap required, opened a new log
I20260809 20:18:02.217625 14038 ts_tablet_manager.cc:1397] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260809 20:18:02.218155 14040 tablet_bootstrap.cc:654] T e0d51271f2c241f9b1816003bca29381 P 5b8af4b6cf364736a928ce119438e9a3: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:02.219393 14040 log.cc:824] T e0d51271f2c241f9b1816003bca29381 P 5b8af4b6cf364736a928ce119438e9a3: Log is configured to *not* fsync() on all Append() calls
I20260809 20:18:02.221210 14037 raft_consensus.cc:348] T e0d51271f2c241f9b1816003bca29381 P 408146e0c09b428b8f9dd510aeecbdc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f1c4c3d4ee43b281427f86818468c1" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38017 } } peers { permanent_uuid: "408146e0c09b428b8f9dd510aeecbdc6" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33467 } } peers { permanent_uuid: "5b8af4b6cf364736a928ce119438e9a3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37061 } }
I20260809 20:18:02.221549 14037 raft_consensus.cc:374] T e0d51271f2c241f9b1816003bca29381 P 408146e0c09b428b8f9dd510aeecbdc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:02.221680 14037 raft_consensus.cc:729] T e0d51271f2c241f9b1816003bca29381 P 408146e0c09b428b8f9dd510aeecbdc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408146e0c09b428b8f9dd510aeecbdc6, State: Initialized, Role: FOLLOWER
I20260809 20:18:02.222193 14040 tablet_bootstrap.cc:492] T e0d51271f2c241f9b1816003bca29381 P 5b8af4b6cf364736a928ce119438e9a3: No bootstrap required, opened a new log
I20260809 20:18:02.222232 14037 consensus_queue.cc:260] T e0d51271f2c241f9b1816003bca29381 P 408146e0c09b428b8f9dd510aeecbdc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f1c4c3d4ee43b281427f86818468c1" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38017 } } peers { permanent_uuid: "408146e0c09b428b8f9dd510aeecbdc6" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33467 } } peers { permanent_uuid: "5b8af4b6cf364736a928ce119438e9a3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37061 } }
I20260809 20:18:02.222434 14040 ts_tablet_manager.cc:1397] T e0d51271f2c241f9b1816003bca29381 P 5b8af4b6cf364736a928ce119438e9a3: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 20:18:02.224642 13998 heartbeater.cc:503] Master 127.5.61.62:40647 was elected leader, sending a full tablet report...
I20260809 20:18:02.225893 14037 ts_tablet_manager.cc:1428] T e0d51271f2c241f9b1816003bca29381 P 408146e0c09b428b8f9dd510aeecbdc6: Time spent starting tablet: real 0.013s user 0.007s sys 0.005s
I20260809 20:18:02.230314 14038 raft_consensus.cc:348] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f1c4c3d4ee43b281427f86818468c1" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38017 } } peers { permanent_uuid: "408146e0c09b428b8f9dd510aeecbdc6" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33467 } } peers { permanent_uuid: "5b8af4b6cf364736a928ce119438e9a3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37061 } }
I20260809 20:18:02.230719 14038 raft_consensus.cc:374] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:02.230818 14038 raft_consensus.cc:729] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f1c4c3d4ee43b281427f86818468c1, State: Initialized, Role: FOLLOWER
I20260809 20:18:02.231431 14038 consensus_queue.cc:260] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f1c4c3d4ee43b281427f86818468c1" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38017 } } peers { permanent_uuid: "408146e0c09b428b8f9dd510aeecbdc6" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33467 } } peers { permanent_uuid: "5b8af4b6cf364736a928ce119438e9a3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37061 } }
I20260809 20:18:02.233086 14038 ts_tablet_manager.cc:1428] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260809 20:18:02.233358 13836 heartbeater.cc:503] Master 127.5.61.62:40647 was elected leader, sending a full tablet report...
I20260809 20:18:02.236011 14040 raft_consensus.cc:348] T e0d51271f2c241f9b1816003bca29381 P 5b8af4b6cf364736a928ce119438e9a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f1c4c3d4ee43b281427f86818468c1" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38017 } } peers { permanent_uuid: "408146e0c09b428b8f9dd510aeecbdc6" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33467 } } peers { permanent_uuid: "5b8af4b6cf364736a928ce119438e9a3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37061 } }
I20260809 20:18:02.236371 14040 raft_consensus.cc:374] T e0d51271f2c241f9b1816003bca29381 P 5b8af4b6cf364736a928ce119438e9a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:02.236496 14040 raft_consensus.cc:729] T e0d51271f2c241f9b1816003bca29381 P 5b8af4b6cf364736a928ce119438e9a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b8af4b6cf364736a928ce119438e9a3, State: Initialized, Role: FOLLOWER
I20260809 20:18:02.237383 14040 consensus_queue.cc:260] T e0d51271f2c241f9b1816003bca29381 P 5b8af4b6cf364736a928ce119438e9a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f1c4c3d4ee43b281427f86818468c1" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38017 } } peers { permanent_uuid: "408146e0c09b428b8f9dd510aeecbdc6" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33467 } } peers { permanent_uuid: "5b8af4b6cf364736a928ce119438e9a3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37061 } }
I20260809 20:18:02.239055 14040 ts_tablet_manager.cc:1428] T e0d51271f2c241f9b1816003bca29381 P 5b8af4b6cf364736a928ce119438e9a3: Time spent starting tablet: real 0.017s user 0.014s sys 0.000s
I20260809 20:18:02.244334 14046 raft_consensus.cc:482] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:18:02.244518 14046 raft_consensus.cc:504] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f1c4c3d4ee43b281427f86818468c1" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38017 } } peers { permanent_uuid: "408146e0c09b428b8f9dd510aeecbdc6" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33467 } } peers { permanent_uuid: "5b8af4b6cf364736a928ce119438e9a3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37061 } }
I20260809 20:18:02.245944 14046 leader_election.cc:288] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 408146e0c09b428b8f9dd510aeecbdc6 (127.5.61.3:33467), 5b8af4b6cf364736a928ce119438e9a3 (127.5.61.1:37061)
I20260809 20:18:02.252188 14036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:02.246326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:57295 (local address 127.5.61.1:37061)
0809 20:18:02.246510 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:02.246515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:02.247249 (+ 734us) server_negotiation.cc:407] Connection header received
0809 20:18:02.247390 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:02.247393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:02.247444 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:02.247510 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:02.248129 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.248772 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.250488 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.251209 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.251363 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:02.251460 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:02.251812 (+ 352us) server_negotiation.cc:299] Negotiation successful
0809 20:18:02.251930 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 20:18:02.252336 14035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:02.246173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:49373 (local address 127.5.61.3:33467)
0809 20:18:02.246387 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:02.246393 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:02.246997 (+ 604us) server_negotiation.cc:407] Connection header received
0809 20:18:02.247187 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:02.247195 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:02.247273 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:02.247369 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:02.248176 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.248816 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.250542 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.251341 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.251486 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:02.251619 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:02.251915 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 20:18:02.252048 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 20:18:02.252358 14049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:02.246356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37061 (local address 127.5.61.2:57295)
0809 20:18:02.247133 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:02.247162 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:02.247280 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:02.247551 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:02.247571 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:02.247601 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:02.247961 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:02.247973 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.248909 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.248913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:02.250321 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:02.250333 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.251406 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.251412 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:02.251660 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:02.251684 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 20:18:02.251778 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 20:18:02.251896 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"mutex_wait_us":24,"thread_start_us":507,"threads_started":1}
I20260809 20:18:02.252801 14048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:02.246055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:33467 (local address 127.5.61.2:49373)
0809 20:18:02.246854 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:02.246879 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:02.247041 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:02.247681 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:02.247684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:02.247710 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:02.248030 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:02.248038 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.248952 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.248956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:02.250383 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:02.250396 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.251583 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.251590 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:02.251741 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:02.251763 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 20:18:02.251891 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 20:18:02.252012 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":3,"thread_start_us":393,"threads_started":1}
I20260809 20:18:02.253782 13945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0d51271f2c241f9b1816003bca29381" candidate_uuid: "07f1c4c3d4ee43b281427f86818468c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408146e0c09b428b8f9dd510aeecbdc6" is_pre_election: true
I20260809 20:18:02.253813 13614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0d51271f2c241f9b1816003bca29381" candidate_uuid: "07f1c4c3d4ee43b281427f86818468c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b8af4b6cf364736a928ce119438e9a3" is_pre_election: true
I20260809 20:18:02.254268 13945 raft_consensus.cc:2454] T e0d51271f2c241f9b1816003bca29381 P 408146e0c09b428b8f9dd510aeecbdc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f1c4c3d4ee43b281427f86818468c1 in term 0.
I20260809 20:18:02.254276 13614 raft_consensus.cc:2454] T e0d51271f2c241f9b1816003bca29381 P 5b8af4b6cf364736a928ce119438e9a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f1c4c3d4ee43b281427f86818468c1 in term 0.
I20260809 20:18:02.254913 13707 leader_election.cc:302] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 07f1c4c3d4ee43b281427f86818468c1, 408146e0c09b428b8f9dd510aeecbdc6, 5b8af4b6cf364736a928ce119438e9a3; no voters:
I20260809 20:18:02.255306 14046 raft_consensus.cc:2781] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:18:02.255417 14046 raft_consensus.cc:482] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:18:02.255486 14046 raft_consensus.cc:3037] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:02.257629 14046 raft_consensus.cc:504] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f1c4c3d4ee43b281427f86818468c1" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38017 } } peers { permanent_uuid: "408146e0c09b428b8f9dd510aeecbdc6" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33467 } } peers { permanent_uuid: "5b8af4b6cf364736a928ce119438e9a3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37061 } }
I20260809 20:18:02.258230 14046 leader_election.cc:288] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1 [CANDIDATE]: Term 1 election: Requested vote from peers 408146e0c09b428b8f9dd510aeecbdc6 (127.5.61.3:33467), 5b8af4b6cf364736a928ce119438e9a3 (127.5.61.1:37061)
I20260809 20:18:02.258595 13945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0d51271f2c241f9b1816003bca29381" candidate_uuid: "07f1c4c3d4ee43b281427f86818468c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408146e0c09b428b8f9dd510aeecbdc6"
I20260809 20:18:02.258759 13614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0d51271f2c241f9b1816003bca29381" candidate_uuid: "07f1c4c3d4ee43b281427f86818468c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b8af4b6cf364736a928ce119438e9a3"
I20260809 20:18:02.258788 13945 raft_consensus.cc:3037] T e0d51271f2c241f9b1816003bca29381 P 408146e0c09b428b8f9dd510aeecbdc6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:02.258948 13614 raft_consensus.cc:3037] T e0d51271f2c241f9b1816003bca29381 P 5b8af4b6cf364736a928ce119438e9a3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:02.261096 13945 raft_consensus.cc:2454] T e0d51271f2c241f9b1816003bca29381 P 408146e0c09b428b8f9dd510aeecbdc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f1c4c3d4ee43b281427f86818468c1 in term 1.
I20260809 20:18:02.261240 13614 raft_consensus.cc:2454] T e0d51271f2c241f9b1816003bca29381 P 5b8af4b6cf364736a928ce119438e9a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f1c4c3d4ee43b281427f86818468c1 in term 1.
I20260809 20:18:02.261530 13708 leader_election.cc:302] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f1c4c3d4ee43b281427f86818468c1, 408146e0c09b428b8f9dd510aeecbdc6; no voters:
I20260809 20:18:02.261812 14046 raft_consensus.cc:2781] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:18:02.262215 14046 raft_consensus.cc:686] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1 [term 1 LEADER]: Becoming Leader. State: Replica: 07f1c4c3d4ee43b281427f86818468c1, State: Running, Role: LEADER
I20260809 20:18:02.262714 14046 consensus_queue.cc:237] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f1c4c3d4ee43b281427f86818468c1" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38017 } } peers { permanent_uuid: "408146e0c09b428b8f9dd510aeecbdc6" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33467 } } peers { permanent_uuid: "5b8af4b6cf364736a928ce119438e9a3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37061 } }
I20260809 20:18:02.267704 12559 catalog_manager.cc:5495] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 07f1c4c3d4ee43b281427f86818468c1 (127.5.61.2). New cstate: current_term: 1 leader_uuid: "07f1c4c3d4ee43b281427f86818468c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f1c4c3d4ee43b281427f86818468c1" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "408146e0c09b428b8f9dd510aeecbdc6" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b8af4b6cf364736a928ce119438e9a3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37061 } health_report { overall_health: UNKNOWN } } }
I20260809 20:18:02.285580 14032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:02.280298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34592 (local address 127.5.61.2:38017)
0809 20:18:02.280502 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:02.280508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:02.280643 (+ 135us) server_negotiation.cc:407] Connection header received
0809 20:18:02.280767 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:02.280771 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:02.280851 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:02.280976 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:18:02.281436 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.282087 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.283360 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.283846 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.283936 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:02.284446 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:18:02.284932 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:18:02.285090 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:02.285169 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 20:18:02.285285 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 20:18:02.293861 13614 raft_consensus.cc:1261] T e0d51271f2c241f9b1816003bca29381 P 5b8af4b6cf364736a928ce119438e9a3 [term 1 FOLLOWER]: Refusing update from remote peer 07f1c4c3d4ee43b281427f86818468c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:18:02.293941 13945 raft_consensus.cc:1261] T e0d51271f2c241f9b1816003bca29381 P 408146e0c09b428b8f9dd510aeecbdc6 [term 1 FOLLOWER]: Refusing update from remote peer 07f1c4c3d4ee43b281427f86818468c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:18:02.294932 14050 consensus_queue.cc:1035] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "408146e0c09b428b8f9dd510aeecbdc6" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:18:02.295336 14046 consensus_queue.cc:1035] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b8af4b6cf364736a928ce119438e9a3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:18:02.325044 14054 mvcc.cc:205] Tried to move back new op lower bound from 7316712170664665088 to 7316712170551705600. Current Snapshot: MvccSnapshot[applied={T|T < 7316712170664665088}]
I20260809 20:18:02.329166 12558 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:18:02.329478 12558 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:18:02.346518 12558 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6798f13fda9446ddae84977ff493a57e: Sending DeleteTablet for 3 replicas of tablet e0d51271f2c241f9b1816003bca29381
I20260809 20:18:02.346947 13592 tablet_service.cc:1514] Processing DeleteTablet for tablet e0d51271f2c241f9b1816003bca29381 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:18:02 UTC) from {username='oryx', principal='oryx/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:39360
I20260809 20:18:02.347824 13921 tablet_service.cc:1514] Processing DeleteTablet for tablet e0d51271f2c241f9b1816003bca29381 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:18:02 UTC) from {username='oryx', principal='oryx/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:42430
I20260809 20:18:02.347874 13758 tablet_service.cc:1514] Processing DeleteTablet for tablet e0d51271f2c241f9b1816003bca29381 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:18:02 UTC) from {username='oryx', principal='oryx/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:34576
I20260809 20:18:02.351424 13591 tablet_service.cc:1514] Processing DeleteTablet for tablet e0d51271f2c241f9b1816003bca29381 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:18:02 UTC) from {username='oryx', principal='oryx/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:39360
I20260809 20:18:02.364168 14067 tablet_replica.cc:330] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1: stopping tablet replica
I20260809 20:18:02.364601 14067 raft_consensus.cc:2229] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1 [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:18:02.365170 14067 raft_consensus.cc:2258] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:18:02.367012 14067 ts_tablet_manager.cc:1905] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:18:02.373030 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 12521
I20260809 20:18:02.374236 12593 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:02.374506 14067 ts_tablet_manager.cc:1918] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:18:02.374696 14067 log.cc:1196] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/e0d51271f2c241f9b1816003bca29381
I20260809 20:18:02.375309 14067 ts_tablet_manager.cc:1939] T e0d51271f2c241f9b1816003bca29381 P 07f1c4c3d4ee43b281427f86818468c1: Deleting consensus metadata
I20260809 20:18:02.390427 14066 tablet_replica.cc:330] T e0d51271f2c241f9b1816003bca29381 P 408146e0c09b428b8f9dd510aeecbdc6: stopping tablet replica
I20260809 20:18:02.392570 14066 raft_consensus.cc:2229] T e0d51271f2c241f9b1816003bca29381 P 408146e0c09b428b8f9dd510aeecbdc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:18:02.394378 14011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:02.354035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46096 (local address 127.5.61.61:42207)
0809 20:18:02.354274 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:02.354283 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:02.370647 (+ 16364us) server_negotiation.cc:407] Connection header received
0809 20:18:02.372877 (+ 2230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:02.372883 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:02.372970 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:02.373053 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:02.377054 (+ 4001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.378630 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.381200 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.381994 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.382107 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:02.386705 (+ 4598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:02.386714 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:02.386737 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:02.386783 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:02.390808 (+ 4025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:02.391718 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:02.391723 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:02.391727 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:02.391922 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:02.392417 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:02.392421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:02.392424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:02.392921 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:02.393152 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:02.393745 (+ 593us) server_negotiation.cc:299] Negotiation successful
0809 20:18:02.394050 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 20:18:02.394374 14066 raft_consensus.cc:2258] T e0d51271f2c241f9b1816003bca29381 P 408146e0c09b428b8f9dd510aeecbdc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:18:02.400955 14066 ts_tablet_manager.cc:1905] T e0d51271f2c241f9b1816003bca29381 P 408146e0c09b428b8f9dd510aeecbdc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:18:02.404050 14065 tablet_replica.cc:330] T e0d51271f2c241f9b1816003bca29381 P 5b8af4b6cf364736a928ce119438e9a3: stopping tablet replica
I20260809 20:18:02.404642 14065 raft_consensus.cc:2229] T e0d51271f2c241f9b1816003bca29381 P 5b8af4b6cf364736a928ce119438e9a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:18:02.405321 14065 raft_consensus.cc:2258] T e0d51271f2c241f9b1816003bca29381 P 5b8af4b6cf364736a928ce119438e9a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:18:02.413378 14066 ts_tablet_manager.cc:1918] T e0d51271f2c241f9b1816003bca29381 P 408146e0c09b428b8f9dd510aeecbdc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:18:02.413605 14066 log.cc:1196] T e0d51271f2c241f9b1816003bca29381 P 408146e0c09b428b8f9dd510aeecbdc6: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/e0d51271f2c241f9b1816003bca29381
I20260809 20:18:02.414006 14016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:02.371180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53016 (local address 127.5.61.60:44599)
0809 20:18:02.371341 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:02.371346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:02.396698 (+ 25352us) server_negotiation.cc:407] Connection header received
0809 20:18:02.396930 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:02.396937 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:02.397043 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:02.397108 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:02.399002 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.400123 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.402103 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.402769 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.402885 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:02.406573 (+ 3688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:02.406581 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:02.406586 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:02.406645 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:02.410430 (+ 3785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:02.411323 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:02.411330 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:02.411333 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:02.411498 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:02.412033 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:02.412038 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:02.412041 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:02.412388 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:02.412596 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:02.413284 (+ 688us) server_negotiation.cc:299] Negotiation successful
0809 20:18:02.413568 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 20:18:02.414253 14066 ts_tablet_manager.cc:1939] T e0d51271f2c241f9b1816003bca29381 P 408146e0c09b428b8f9dd510aeecbdc6: Deleting consensus metadata
I20260809 20:18:02.416790 14065 ts_tablet_manager.cc:1905] T e0d51271f2c241f9b1816003bca29381 P 5b8af4b6cf364736a928ce119438e9a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:18:02.426214 14065 ts_tablet_manager.cc:1918] T e0d51271f2c241f9b1816003bca29381 P 5b8af4b6cf364736a928ce119438e9a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:18:02.426404 14065 log.cc:1196] T e0d51271f2c241f9b1816003bca29381 P 5b8af4b6cf364736a928ce119438e9a3: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/e0d51271f2c241f9b1816003bca29381
I20260809 20:18:02.426977 14065 ts_tablet_manager.cc:1939] T e0d51271f2c241f9b1816003bca29381 P 5b8af4b6cf364736a928ce119438e9a3: Deleting consensus metadata
I20260809 20:18:02.546762 12543 catalog_manager.cc:4841] TS 408146e0c09b428b8f9dd510aeecbdc6 (127.5.61.3:33467): tablet e0d51271f2c241f9b1816003bca29381 (table test-table [id=090945c5b9704804a0a8020e4fef84cf]) successfully deleted
I20260809 20:18:02.547536 12542 catalog_manager.cc:4841] TS 07f1c4c3d4ee43b281427f86818468c1 (127.5.61.2:38017): tablet e0d51271f2c241f9b1816003bca29381 (table test-table [id=090945c5b9704804a0a8020e4fef84cf]) successfully deleted
W20260809 20:18:02.547781 12542 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.5.61.2:38017, user_credentials={real_user=slave}} blocked reactor thread for 170630us
W20260809 20:18:02.548807 12542 catalog_manager.cc:4808] TS 5b8af4b6cf364736a928ce119438e9a3 (127.5.61.1:37061): delete failed for tablet e0d51271f2c241f9b1816003bca29381 because the tablet was not found. No further retry: Not found: Tablet not found: e0d51271f2c241f9b1816003bca29381
I20260809 20:18:02.549203 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 12521
I20260809 20:18:02.608043 14016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:02.589199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53026 (local address 127.5.61.60:44599)
0809 20:18:02.589479 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:02.589488 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:02.590663 (+ 1175us) server_negotiation.cc:407] Connection header received
0809 20:18:02.590745 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:02.590750 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:02.590835 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:02.590976 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:02.593300 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.594478 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.596312 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.596982 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.597114 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:02.600707 (+ 3593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:02.600713 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:02.600718 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:02.600759 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:02.604556 (+ 3797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:02.605458 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:02.605464 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:02.605467 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:02.605627 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:02.606035 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:02.606040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:02.606043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:02.606388 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:02.606575 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:02.607423 (+ 848us) server_negotiation.cc:299] Negotiation successful
0809 20:18:02.607680 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 20:18:02.608151 14011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:02.590569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46100 (local address 127.5.61.61:42207)
0809 20:18:02.590823 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:02.590829 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:02.591787 (+ 958us) server_negotiation.cc:407] Connection header received
0809 20:18:02.591954 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:02.591959 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:02.592051 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:02.592156 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:02.594296 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.595533 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.597598 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:02.598255 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:02.598401 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:02.601420 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:02.601429 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:02.601436 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:02.601482 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:02.604715 (+ 3233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:02.605614 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:02.605618 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:02.605621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:02.605766 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:02.606626 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:02.606629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:02.606631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:02.606911 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:02.607168 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:02.607517 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 20:18:02.607776 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260809 20:18:03.329331 14076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.328650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.5.61.2:36877)
0809 20:18:03.329066 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:03.329174 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":213,"threads_started":1}
W20260809 20:18:03.329855 13836 heartbeater.cc:644] Failed to heartbeat to 127.5.61.62:40647 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
I20260809 20:18:03.347303 14077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.346444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.5.61.3:39219)
0809 20:18:03.346896 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:03.347018 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":237,"threads_started":1}
W20260809 20:18:03.347811 13998 heartbeater.cc:644] Failed to heartbeat to 127.5.61.62:40647 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
I20260809 20:18:03.348201 14078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.347394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.5.61.1:60729)
0809 20:18:03.347944 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:03.348069 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":308,"threads_started":1}
W20260809 20:18:03.348580 13668 heartbeater.cc:644] Failed to heartbeat to 127.5.61.62:40647 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
I20260809 20:18:03.843897 14079 raft_consensus.cc:482] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6798f13fda9446ddae84977ff493a57e)
I20260809 20:18:03.844198 14079 raft_consensus.cc:504] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: false } }
I20260809 20:18:03.845579 14079 leader_election.cc:288] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6798f13fda9446ddae84977ff493a57e (127.5.61.62:40647), 32de6f510b03403ead9eca45eae81f33 (127.5.61.60:44599)
I20260809 20:18:03.846128 14080 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6798f13fda9446ddae84977ff493a57e)
I20260809 20:18:03.846457 14080 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: false } }
I20260809 20:18:03.847007 14081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.845784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.0.0.1:55548)
0809 20:18:03.846408 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:03.846622 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":409,"threads_started":1}
W20260809 20:18:03.847295 13238 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
I20260809 20:18:03.855296 14086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.850707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.0.0.1:55552)
0809 20:18:03.854725 (+ 4018us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:03.854948 (+ 223us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3851,"thread_start_us":3701,"threads_started":1}
I20260809 20:18:03.856087 14080 leader_election.cc:288] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6798f13fda9446ddae84977ff493a57e (127.5.61.62:40647), e272cdd4371e455c91ad74c33271afdd (127.5.61.61:42207)
I20260809 20:18:03.857628 14081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.846184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:44599 (local address 127.0.0.1:53040)
0809 20:18:03.847225 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:03.847242 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:03.847395 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:03.848492 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:03.848500 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:03.848537 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:03.848926 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:03.848940 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:03.851487 (+ 2547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:03.851495 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:03.854118 (+ 2623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:03.854139 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:03.856563 (+ 2424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:03.856572 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:03.856846 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:03.856917 (+ 71us) client_negotiation.cc:769] Sending connection context
0809 20:18:03.857005 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 20:18:03.857196 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":958,"threads_started":1}
I20260809 20:18:03.858757 14082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.846092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53040 (local address 127.5.61.60:44599)
0809 20:18:03.848033 (+ 1941us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:03.848042 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:03.848073 (+ 31us) server_negotiation.cc:407] Connection header received
0809 20:18:03.848164 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:03.848173 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:03.848273 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:03.849359 (+ 1086us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:03.849581 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:03.851232 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:03.854465 (+ 3233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:03.856306 (+ 1841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:03.857897 (+ 1591us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:03.858047 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:03.858113 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 20:18:03.858295 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1785,"thread_start_us":1627,"threads_started":1}
W20260809 20:18:03.861532 13454 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
I20260809 20:18:03.862270 14084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.861543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.0.0.1:55568)
0809 20:18:03.861781 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:03.862002 (+ 221us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 20:18:03.862438 13238 leader_election.cc:334] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6798f13fda9446ddae84977ff493a57e (127.5.61.62:40647): Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
I20260809 20:18:03.862865 13481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e272cdd4371e455c91ad74c33271afdd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "32de6f510b03403ead9eca45eae81f33" is_pre_election: true
I20260809 20:18:03.863188 13481 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e272cdd4371e455c91ad74c33271afdd in term 1.
I20260809 20:18:03.863972 13240 leader_election.cc:302] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32de6f510b03403ead9eca45eae81f33, e272cdd4371e455c91ad74c33271afdd; no voters: 6798f13fda9446ddae84977ff493a57e
I20260809 20:18:03.864177 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.863617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.0.0.1:55570)
0809 20:18:03.863817 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:03.863980 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 20:18:03.864329 14079 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 20:18:03.864441 14079 raft_consensus.cc:482] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6798f13fda9446ddae84977ff493a57e)
W20260809 20:18:03.864379 13454 leader_election.cc:334] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6798f13fda9446ddae84977ff493a57e (127.5.61.62:40647): Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
I20260809 20:18:03.864526 14079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 1 FOLLOWER]: Advancing to term 2
I20260809 20:18:03.867105 14088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.855059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46102 (local address 127.5.61.61:42207)
0809 20:18:03.860703 (+ 5644us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:03.860710 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:03.860743 (+ 33us) server_negotiation.cc:407] Connection header received
0809 20:18:03.860800 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:03.860806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:03.860902 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:03.861010 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:03.861645 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:03.862956 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:03.865320 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:03.866191 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:03.866302 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:03.866408 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:03.866769 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 20:18:03.866876 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5510,"thread_start_us":5294,"threads_started":1}
I20260809 20:18:03.867192 14086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.855021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:42207 (local address 127.0.0.1:46102)
0809 20:18:03.855540 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:03.855575 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:03.855755 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:03.861195 (+ 5440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:03.861202 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:03.861240 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:03.861470 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:03.861481 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:03.863134 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:03.863142 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:03.865127 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:03.865141 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:03.866407 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:03.866416 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:03.866603 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:03.866636 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 20:18:03.866739 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 20:18:03.866893 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":6231,"threads_started":1}
I20260809 20:18:03.867537 14079 raft_consensus.cc:504] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: false } }
I20260809 20:18:03.867892 13268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "32de6f510b03403ead9eca45eae81f33" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e272cdd4371e455c91ad74c33271afdd" is_pre_election: true
I20260809 20:18:03.868332 14079 leader_election.cc:288] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [CANDIDATE]: Term 2 election: Requested vote from peers 6798f13fda9446ddae84977ff493a57e (127.5.61.62:40647), 32de6f510b03403ead9eca45eae81f33 (127.5.61.60:44599)
I20260809 20:18:03.868359 13268 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 32de6f510b03403ead9eca45eae81f33 in current term 2: Already voted for candidate e272cdd4371e455c91ad74c33271afdd in this term.
I20260809 20:18:03.868743 14084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.868278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.0.0.1:55574)
0809 20:18:03.868524 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:03.868614 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260809 20:18:03.868835 13481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e272cdd4371e455c91ad74c33271afdd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "32de6f510b03403ead9eca45eae81f33"
I20260809 20:18:03.869032 13454 leader_election.cc:302] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32de6f510b03403ead9eca45eae81f33; no voters: 6798f13fda9446ddae84977ff493a57e, e272cdd4371e455c91ad74c33271afdd
I20260809 20:18:03.869057 13481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [term 1 FOLLOWER]: Advancing to term 2
I20260809 20:18:03.869776 14084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.869372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.0.0.1:55578)
0809 20:18:03.869535 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:03.869619 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 20:18:03.869962 13238 leader_election.cc:334] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6798f13fda9446ddae84977ff493a57e (127.5.61.62:40647): Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
I20260809 20:18:03.871555 13481 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e272cdd4371e455c91ad74c33271afdd in term 2.
I20260809 20:18:03.871870 14080 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 20:18:03.871965 13240 leader_election.cc:302] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32de6f510b03403ead9eca45eae81f33, e272cdd4371e455c91ad74c33271afdd; no voters: 6798f13fda9446ddae84977ff493a57e
I20260809 20:18:03.872334 14079 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 2 FOLLOWER]: Leader election won for term 2
I20260809 20:18:03.872768 14079 raft_consensus.cc:686] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [term 2 LEADER]: Becoming Leader. State: Replica: e272cdd4371e455c91ad74c33271afdd, State: Running, Role: LEADER
I20260809 20:18:03.873119 14079 consensus_queue.cc:237] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: false } }
I20260809 20:18:03.874878 14090 sys_catalog.cc:455] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [sys.catalog]: SysCatalogTable state changed. Reason: New leader e272cdd4371e455c91ad74c33271afdd. Latest consensus state: current_term: 2 leader_uuid: "e272cdd4371e455c91ad74c33271afdd" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: false } } }
I20260809 20:18:03.875185 14090 sys_catalog.cc:458] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [sys.catalog]: This master's current role is: LEADER
I20260809 20:18:03.875907 14092 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:18:03.879732 14092 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:18:03.880564 14092 catalog_manager.cc:1259] Loaded cluster ID: 19ac0cc23316463f8eba2700c4b6ac78
I20260809 20:18:03.880666 14092 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:18:03.881800 14092 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:18:03.882606 14092 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd: Loaded TSK: 0
I20260809 20:18:03.883509 14092 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:18:03.915335 13257 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46100:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:18:03.923136 13257 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:18:03.928320 14084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.927732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.0.0.1:55590)
0809 20:18:03.928047 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:03.928141 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 20:18:03.928298 13481 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [term 2 FOLLOWER]: Refusing update from remote peer e272cdd4371e455c91ad74c33271afdd: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 20:18:03.929253 14090 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 20:18:03.929519 14084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.929054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.0.0.1:55592)
0809 20:18:03.929250 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:03.929330 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 20:18:03.929802 13238 consensus_peers.cc:487] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd -> Peer 6798f13fda9446ddae84977ff493a57e (127.5.61.62:40647): Couldn't send request to peer 6798f13fda9446ddae84977ff493a57e. Status: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 20:18:03.932096 14080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e272cdd4371e455c91ad74c33271afdd. Latest consensus state: current_term: 2 leader_uuid: "e272cdd4371e455c91ad74c33271afdd" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: false } } }
I20260809 20:18:03.932545 14080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:18:03.934152 14079 sys_catalog.cc:455] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "e272cdd4371e455c91ad74c33271afdd" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: false } } }
I20260809 20:18:03.934505 14079 sys_catalog.cc:458] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd [sys.catalog]: This master's current role is: LEADER
I20260809 20:18:03.935261 14080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "e272cdd4371e455c91ad74c33271afdd" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6798f13fda9446ddae84977ff493a57e" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40647 } } peers { permanent_uuid: "e272cdd4371e455c91ad74c33271afdd" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42207 } attrs { promote: false } } peers { permanent_uuid: "32de6f510b03403ead9eca45eae81f33" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 44599 } attrs { promote: false } } }
I20260809 20:18:03.935592 14080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32de6f510b03403ead9eca45eae81f33 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:18:03.935683 14094 mvcc.cc:205] Tried to move back new op lower bound from 7316712177362681856 to 7316712177147604992. Current Snapshot: MvccSnapshot[applied={T|T < 7316712177362681856}]
I20260809 20:18:03.952062 14081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.943809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37061 (local address 127.0.0.1:39364)
0809 20:18:03.943937 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:03.943950 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:03.944062 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:03.945350 (+ 1288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:03.945356 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:03.945396 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:03.945643 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:03.945662 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:03.947217 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:03.947222 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:03.949266 (+ 2044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:03.949281 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:03.950870 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:03.950878 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:03.951146 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:03.951203 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 20:18:03.951319 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 20:18:03.951518 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":3}
I20260809 20:18:03.952135 14101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.944176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39364 (local address 127.5.61.1:37061)
0809 20:18:03.945007 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:03.945012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:03.945027 (+ 15us) server_negotiation.cc:407] Connection header received
0809 20:18:03.945086 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:03.945089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:03.945186 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:03.946147 (+ 961us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:03.946335 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:03.947064 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:03.949456 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:03.950662 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:03.950788 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:03.950914 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:03.951668 (+ 754us) server_negotiation.cc:299] Negotiation successful
0809 20:18:03.951830 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":723,"thread_start_us":615,"threads_started":1}
I20260809 20:18:03.952399 14084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.943391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:38017 (local address 127.0.0.1:34594)
0809 20:18:03.943544 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:03.943584 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:03.943727 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:03.945529 (+ 1802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:03.945533 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:03.945559 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:03.945931 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:03.945943 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:03.947594 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:03.947600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:03.949759 (+ 2159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:03.949772 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:03.951442 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:03.951448 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:03.951683 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:03.951720 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 20:18:03.951809 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 20:18:03.951946 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":19}
I20260809 20:18:03.952996 14100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.943487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34594 (local address 127.5.61.2:38017)
0809 20:18:03.944816 (+ 1329us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:03.944825 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:03.944848 (+ 23us) server_negotiation.cc:407] Connection header received
0809 20:18:03.944936 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:03.944944 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:03.945042 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:03.945160 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:03.946099 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:03.947419 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:03.949967 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:03.951196 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:03.951348 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:03.951467 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:03.952498 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0809 20:18:03.952648 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1191,"thread_start_us":1096,"threads_started":1}
I20260809 20:18:03.953467 13592 tablet_service.cc:1467] Processing CreateTablet for tablet d728c7f604fb41178577fc079a9cef06 (DEFAULT_TABLE table=test-table [id=bda0772a52384c71a06aaa92fa27a89d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:18:03.953758 13758 tablet_service.cc:1467] Processing CreateTablet for tablet d728c7f604fb41178577fc079a9cef06 (DEFAULT_TABLE table=test-table [id=bda0772a52384c71a06aaa92fa27a89d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:18:03.954037 13592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d728c7f604fb41178577fc079a9cef06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:03.954245 13758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d728c7f604fb41178577fc079a9cef06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:03.959059 14104 tablet_bootstrap.cc:492] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1: Bootstrap starting.
I20260809 20:18:03.960649 14104 tablet_bootstrap.cc:654] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:03.962968 14104 tablet_bootstrap.cc:492] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1: No bootstrap required, opened a new log
I20260809 20:18:03.963136 14104 ts_tablet_manager.cc:1397] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 20:18:03.964067 14106 tablet_bootstrap.cc:492] T d728c7f604fb41178577fc079a9cef06 P 5b8af4b6cf364736a928ce119438e9a3: Bootstrap starting.
I20260809 20:18:03.964015 14104 raft_consensus.cc:348] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f1c4c3d4ee43b281427f86818468c1" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38017 } } peers { permanent_uuid: "5b8af4b6cf364736a928ce119438e9a3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37061 } } peers { permanent_uuid: "408146e0c09b428b8f9dd510aeecbdc6" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33467 } }
I20260809 20:18:03.964253 14104 raft_consensus.cc:374] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:03.964339 14104 raft_consensus.cc:729] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f1c4c3d4ee43b281427f86818468c1, State: Initialized, Role: FOLLOWER
I20260809 20:18:03.964555 14104 consensus_queue.cc:260] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f1c4c3d4ee43b281427f86818468c1" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38017 } } peers { permanent_uuid: "5b8af4b6cf364736a928ce119438e9a3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37061 } } peers { permanent_uuid: "408146e0c09b428b8f9dd510aeecbdc6" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33467 } }
I20260809 20:18:03.965452 14104 ts_tablet_manager.cc:1428] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 20:18:03.966390 14106 tablet_bootstrap.cc:654] T d728c7f604fb41178577fc079a9cef06 P 5b8af4b6cf364736a928ce119438e9a3: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:03.973652 14106 tablet_bootstrap.cc:492] T d728c7f604fb41178577fc079a9cef06 P 5b8af4b6cf364736a928ce119438e9a3: No bootstrap required, opened a new log
I20260809 20:18:03.973802 14106 ts_tablet_manager.cc:1397] T d728c7f604fb41178577fc079a9cef06 P 5b8af4b6cf364736a928ce119438e9a3: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 20:18:03.974673 14106 raft_consensus.cc:348] T d728c7f604fb41178577fc079a9cef06 P 5b8af4b6cf364736a928ce119438e9a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f1c4c3d4ee43b281427f86818468c1" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38017 } } peers { permanent_uuid: "5b8af4b6cf364736a928ce119438e9a3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37061 } } peers { permanent_uuid: "408146e0c09b428b8f9dd510aeecbdc6" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33467 } }
I20260809 20:18:03.974902 14106 raft_consensus.cc:374] T d728c7f604fb41178577fc079a9cef06 P 5b8af4b6cf364736a928ce119438e9a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:03.975013 14106 raft_consensus.cc:729] T d728c7f604fb41178577fc079a9cef06 P 5b8af4b6cf364736a928ce119438e9a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b8af4b6cf364736a928ce119438e9a3, State: Initialized, Role: FOLLOWER
I20260809 20:18:03.975297 14106 consensus_queue.cc:260] T d728c7f604fb41178577fc079a9cef06 P 5b8af4b6cf364736a928ce119438e9a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f1c4c3d4ee43b281427f86818468c1" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38017 } } peers { permanent_uuid: "5b8af4b6cf364736a928ce119438e9a3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37061 } } peers { permanent_uuid: "408146e0c09b428b8f9dd510aeecbdc6" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33467 } }
I20260809 20:18:03.975500 14108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.966124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.5.61.2:35529)
0809 20:18:03.973506 (+ 7382us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:03.974809 (+ 1303us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7258,"thread_start_us":7106,"threads_started":1}
I20260809 20:18:03.976166 13837 heartbeater.cc:503] Master 127.5.61.61:42207 was elected leader, sending a full tablet report...
I20260809 20:18:03.976608 14106 ts_tablet_manager.cc:1428] T d728c7f604fb41178577fc079a9cef06 P 5b8af4b6cf364736a928ce119438e9a3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 20:18:03.977349 14108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.976854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.5.61.2:60611)
0809 20:18:03.977135 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:03.977217 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
I20260809 20:18:03.978241 14084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.944212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:33467 (local address 127.0.0.1:42440)
0809 20:18:03.952683 (+ 8471us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:03.952701 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:03.952818 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:03.971699 (+ 18881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:03.971708 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:03.971778 (+ 70us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:03.972035 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:03.972053 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:03.973937 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:03.973942 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:03.975907 (+ 1965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:03.975922 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:03.977437 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:03.977445 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:03.977672 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:03.977709 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 20:18:03.977792 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 20:18:03.977953 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8390,"thread_start_us":28673,"threads_started":1}
I20260809 20:18:03.978777 14103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.944665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42440 (local address 127.5.61.3:33467)
0809 20:18:03.971078 (+ 26413us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:03.971088 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:03.971117 (+ 29us) server_negotiation.cc:407] Connection header received
0809 20:18:03.971212 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:03.971221 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:03.971347 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:03.972231 (+ 884us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:03.972484 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:03.973754 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:03.976114 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:03.977257 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:03.978064 (+ 807us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:03.978176 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:03.978245 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 20:18:03.978476 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26235,"thread_start_us":26809,"threads_started":1}
I20260809 20:18:03.979771 13921 tablet_service.cc:1467] Processing CreateTablet for tablet d728c7f604fb41178577fc079a9cef06 (DEFAULT_TABLE table=test-table [id=bda0772a52384c71a06aaa92fa27a89d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:18:03.980180 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.977786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.5.61.1:60549)
0809 20:18:03.979784 (+ 1998us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:03.979936 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1857,"thread_start_us":1706,"threads_started":1}
I20260809 20:18:03.980263 13921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d728c7f604fb41178577fc079a9cef06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:03.981231 13669 heartbeater.cc:503] Master 127.5.61.61:42207 was elected leader, sending a full tablet report...
I20260809 20:18:03.981288 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.980886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.5.61.1:60157)
0809 20:18:03.981051 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:03.981135 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 20:18:03.986194 14113 tablet_bootstrap.cc:492] T d728c7f604fb41178577fc079a9cef06 P 408146e0c09b428b8f9dd510aeecbdc6: Bootstrap starting.
I20260809 20:18:03.987533 14113 tablet_bootstrap.cc:654] T d728c7f604fb41178577fc079a9cef06 P 408146e0c09b428b8f9dd510aeecbdc6: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:03.989594 14113 tablet_bootstrap.cc:492] T d728c7f604fb41178577fc079a9cef06 P 408146e0c09b428b8f9dd510aeecbdc6: No bootstrap required, opened a new log
I20260809 20:18:03.989768 14113 ts_tablet_manager.cc:1397] T d728c7f604fb41178577fc079a9cef06 P 408146e0c09b428b8f9dd510aeecbdc6: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 20:18:03.990435 14113 raft_consensus.cc:348] T d728c7f604fb41178577fc079a9cef06 P 408146e0c09b428b8f9dd510aeecbdc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f1c4c3d4ee43b281427f86818468c1" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38017 } } peers { permanent_uuid: "5b8af4b6cf364736a928ce119438e9a3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37061 } } peers { permanent_uuid: "408146e0c09b428b8f9dd510aeecbdc6" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33467 } }
I20260809 20:18:03.990604 14113 raft_consensus.cc:374] T d728c7f604fb41178577fc079a9cef06 P 408146e0c09b428b8f9dd510aeecbdc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:03.990715 14113 raft_consensus.cc:729] T d728c7f604fb41178577fc079a9cef06 P 408146e0c09b428b8f9dd510aeecbdc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408146e0c09b428b8f9dd510aeecbdc6, State: Initialized, Role: FOLLOWER
I20260809 20:18:03.990994 14113 consensus_queue.cc:260] T d728c7f604fb41178577fc079a9cef06 P 408146e0c09b428b8f9dd510aeecbdc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f1c4c3d4ee43b281427f86818468c1" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38017 } } peers { permanent_uuid: "5b8af4b6cf364736a928ce119438e9a3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37061 } } peers { permanent_uuid: "408146e0c09b428b8f9dd510aeecbdc6" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33467 } }
I20260809 20:18:03.992112 14113 ts_tablet_manager.cc:1428] T d728c7f604fb41178577fc079a9cef06 P 408146e0c09b428b8f9dd510aeecbdc6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 20:18:03.993042 14117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.992048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.5.61.3:34655)
0809 20:18:03.992666 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:03.992778 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":345,"threads_started":1}
I20260809 20:18:03.994239 13999 heartbeater.cc:503] Master 127.5.61.61:42207 was elected leader, sending a full tablet report...
I20260809 20:18:03.994760 14117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:03.994392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.5.61.3:59067)
0809 20:18:03.994523 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:03.994603 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 20:18:04.029412 14107 raft_consensus.cc:482] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:18:04.029666 14107 raft_consensus.cc:504] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f1c4c3d4ee43b281427f86818468c1" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38017 } } peers { permanent_uuid: "5b8af4b6cf364736a928ce119438e9a3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37061 } } peers { permanent_uuid: "408146e0c09b428b8f9dd510aeecbdc6" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33467 } }
I20260809 20:18:04.030478 14107 leader_election.cc:288] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b8af4b6cf364736a928ce119438e9a3 (127.5.61.1:37061), 408146e0c09b428b8f9dd510aeecbdc6 (127.5.61.3:33467)
I20260809 20:18:04.031109 13614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d728c7f604fb41178577fc079a9cef06" candidate_uuid: "07f1c4c3d4ee43b281427f86818468c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b8af4b6cf364736a928ce119438e9a3" is_pre_election: true
I20260809 20:18:04.031108 13945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d728c7f604fb41178577fc079a9cef06" candidate_uuid: "07f1c4c3d4ee43b281427f86818468c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408146e0c09b428b8f9dd510aeecbdc6" is_pre_election: true
I20260809 20:18:04.031368 13945 raft_consensus.cc:2454] T d728c7f604fb41178577fc079a9cef06 P 408146e0c09b428b8f9dd510aeecbdc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f1c4c3d4ee43b281427f86818468c1 in term 0.
I20260809 20:18:04.031380 13614 raft_consensus.cc:2454] T d728c7f604fb41178577fc079a9cef06 P 5b8af4b6cf364736a928ce119438e9a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f1c4c3d4ee43b281427f86818468c1 in term 0.
I20260809 20:18:04.031894 13707 leader_election.cc:302] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f1c4c3d4ee43b281427f86818468c1, 5b8af4b6cf364736a928ce119438e9a3; no voters:
I20260809 20:18:04.032207 14107 raft_consensus.cc:2781] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:18:04.032358 14107 raft_consensus.cc:482] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:18:04.032470 14107 raft_consensus.cc:3037] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:04.034502 14107 raft_consensus.cc:504] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f1c4c3d4ee43b281427f86818468c1" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38017 } } peers { permanent_uuid: "5b8af4b6cf364736a928ce119438e9a3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37061 } } peers { permanent_uuid: "408146e0c09b428b8f9dd510aeecbdc6" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33467 } }
I20260809 20:18:04.034997 14107 leader_election.cc:288] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1 [CANDIDATE]: Term 1 election: Requested vote from peers 5b8af4b6cf364736a928ce119438e9a3 (127.5.61.1:37061), 408146e0c09b428b8f9dd510aeecbdc6 (127.5.61.3:33467)
I20260809 20:18:04.035531 13945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d728c7f604fb41178577fc079a9cef06" candidate_uuid: "07f1c4c3d4ee43b281427f86818468c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408146e0c09b428b8f9dd510aeecbdc6"
I20260809 20:18:04.035531 13614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d728c7f604fb41178577fc079a9cef06" candidate_uuid: "07f1c4c3d4ee43b281427f86818468c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b8af4b6cf364736a928ce119438e9a3"
I20260809 20:18:04.035768 13614 raft_consensus.cc:3037] T d728c7f604fb41178577fc079a9cef06 P 5b8af4b6cf364736a928ce119438e9a3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:04.035780 13945 raft_consensus.cc:3037] T d728c7f604fb41178577fc079a9cef06 P 408146e0c09b428b8f9dd510aeecbdc6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:04.038776 13614 raft_consensus.cc:2454] T d728c7f604fb41178577fc079a9cef06 P 5b8af4b6cf364736a928ce119438e9a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f1c4c3d4ee43b281427f86818468c1 in term 1.
I20260809 20:18:04.038780 13945 raft_consensus.cc:2454] T d728c7f604fb41178577fc079a9cef06 P 408146e0c09b428b8f9dd510aeecbdc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f1c4c3d4ee43b281427f86818468c1 in term 1.
I20260809 20:18:04.039203 13708 leader_election.cc:302] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f1c4c3d4ee43b281427f86818468c1, 408146e0c09b428b8f9dd510aeecbdc6; no voters:
I20260809 20:18:04.039499 14107 raft_consensus.cc:2781] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:18:04.039969 14107 raft_consensus.cc:686] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1 [term 1 LEADER]: Becoming Leader. State: Replica: 07f1c4c3d4ee43b281427f86818468c1, State: Running, Role: LEADER
I20260809 20:18:04.040313 14107 consensus_queue.cc:237] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f1c4c3d4ee43b281427f86818468c1" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38017 } } peers { permanent_uuid: "5b8af4b6cf364736a928ce119438e9a3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37061 } } peers { permanent_uuid: "408146e0c09b428b8f9dd510aeecbdc6" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33467 } }
I20260809 20:18:04.041082 14108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:04.040588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.5.61.2:35693)
0809 20:18:04.040756 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:04.040890 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 20:18:04.044061 13257 catalog_manager.cc:5495] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 07f1c4c3d4ee43b281427f86818468c1 (127.5.61.2). New cstate: current_term: 1 leader_uuid: "07f1c4c3d4ee43b281427f86818468c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f1c4c3d4ee43b281427f86818468c1" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 38017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b8af4b6cf364736a928ce119438e9a3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "408146e0c09b428b8f9dd510aeecbdc6" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33467 } health_report { overall_health: UNKNOWN } } }
I20260809 20:18:04.062677 14100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:04.057678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34598 (local address 127.5.61.2:38017)
0809 20:18:04.057904 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:04.057910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:04.058035 (+ 125us) server_negotiation.cc:407] Connection header received
0809 20:18:04.058255 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:04.058266 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:04.058344 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:04.058474 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:18:04.058863 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:04.059653 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:04.060946 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:04.061363 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:04.061503 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:04.061969 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:18:04.062067 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:18:04.062166 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:04.062324 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 20:18:04.062403 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 20:18:04.065977 13945 raft_consensus.cc:1261] T d728c7f604fb41178577fc079a9cef06 P 408146e0c09b428b8f9dd510aeecbdc6 [term 1 FOLLOWER]: Refusing update from remote peer 07f1c4c3d4ee43b281427f86818468c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:18:04.066048 13614 raft_consensus.cc:1261] T d728c7f604fb41178577fc079a9cef06 P 5b8af4b6cf364736a928ce119438e9a3 [term 1 FOLLOWER]: Refusing update from remote peer 07f1c4c3d4ee43b281427f86818468c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:18:04.066727 14119 consensus_queue.cc:1035] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b8af4b6cf364736a928ce119438e9a3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:18:04.067101 14107 consensus_queue.cc:1035] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "408146e0c09b428b8f9dd510aeecbdc6" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:18:04.069931 14117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:04.069230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.5.61.3:57625)
0809 20:18:04.069452 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:04.069652 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 20:18:04.069963 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:04.069277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.5.61.1:43535)
0809 20:18:04.069566 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:04.069709 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260809 20:18:04.075399 14117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:04.074781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.5.61.3:50687)
0809 20:18:04.075048 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:04.075169 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260809 20:18:04.079154 14123 mvcc.cc:205] Tried to move back new op lower bound from 7316712177927450624 to 7316712177831862272. Current Snapshot: MvccSnapshot[applied={T|T < 7316712177927450624}]
W20260809 20:18:04.079366 13670 tablet.cc:2369] T d728c7f604fb41178577fc079a9cef06 P 5b8af4b6cf364736a928ce119438e9a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:18:04.085328 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:04.084716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.5.61.1:34457)
0809 20:18:04.084977 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:04.085088 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260809 20:18:04.088600 14108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:04.083792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.5.61.2:35405)
0809 20:18:04.088316 (+ 4524us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:04.088430 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4399}
I20260809 20:18:04.091940 14108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:04.091418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.5.61.2:49523)
0809 20:18:04.091629 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:04.091767 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 20:18:04.092025 13255 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:18:04.092515 13255 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:18:04.098533 13255 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e272cdd4371e455c91ad74c33271afdd: Sending DeleteTablet for 3 replicas of tablet d728c7f604fb41178577fc079a9cef06
I20260809 20:18:04.099601 13758 tablet_service.cc:1514] Processing DeleteTablet for tablet d728c7f604fb41178577fc079a9cef06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:18:04 UTC) from {username='oryx', principal='oryx/127.5.61.61@KRBTEST.COM'} at 127.0.0.1:34594
I20260809 20:18:04.099625 13592 tablet_service.cc:1514] Processing DeleteTablet for tablet d728c7f604fb41178577fc079a9cef06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:18:04 UTC) from {username='oryx', principal='oryx/127.5.61.61@KRBTEST.COM'} at 127.0.0.1:39364
I20260809 20:18:04.100052 13921 tablet_service.cc:1514] Processing DeleteTablet for tablet d728c7f604fb41178577fc079a9cef06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:18:04 UTC) from {username='oryx', principal='oryx/127.5.61.61@KRBTEST.COM'} at 127.0.0.1:42440
I20260809 20:18:04.101006 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 13450
I20260809 20:18:04.104060 14132 tablet_replica.cc:330] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1: stopping tablet replica
I20260809 20:18:04.104274 14132 raft_consensus.cc:2229] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1 [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:18:04.104595 14132 raft_consensus.cc:2258] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:18:04.105463 14132 ts_tablet_manager.cc:1905] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:18:04.111127 14132 ts_tablet_manager.cc:1918] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:18:04.111279 14132 log.cc:1196] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/d728c7f604fb41178577fc079a9cef06
I20260809 20:18:04.111719 14132 ts_tablet_manager.cc:1939] T d728c7f604fb41178577fc079a9cef06 P 07f1c4c3d4ee43b281427f86818468c1: Deleting consensus metadata
I20260809 20:18:04.112818 13237 catalog_manager.cc:4841] TS 07f1c4c3d4ee43b281427f86818468c1 (127.5.61.2:38017): tablet d728c7f604fb41178577fc079a9cef06 (table test-table [id=bda0772a52384c71a06aaa92fa27a89d]) successfully deleted
I20260809 20:18:04.113379 14134 tablet_replica.cc:330] T d728c7f604fb41178577fc079a9cef06 P 408146e0c09b428b8f9dd510aeecbdc6: stopping tablet replica
I20260809 20:18:04.113569 14134 raft_consensus.cc:2229] T d728c7f604fb41178577fc079a9cef06 P 408146e0c09b428b8f9dd510aeecbdc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:18:04.113860 14134 raft_consensus.cc:2258] T d728c7f604fb41178577fc079a9cef06 P 408146e0c09b428b8f9dd510aeecbdc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:18:04.114408 14133 tablet_replica.cc:330] T d728c7f604fb41178577fc079a9cef06 P 5b8af4b6cf364736a928ce119438e9a3: stopping tablet replica
I20260809 20:18:04.114671 14133 raft_consensus.cc:2229] T d728c7f604fb41178577fc079a9cef06 P 5b8af4b6cf364736a928ce119438e9a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:18:04.115021 14133 raft_consensus.cc:2258] T d728c7f604fb41178577fc079a9cef06 P 5b8af4b6cf364736a928ce119438e9a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:18:04.115020 14134 ts_tablet_manager.cc:1905] T d728c7f604fb41178577fc079a9cef06 P 408146e0c09b428b8f9dd510aeecbdc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:18:04.116235 14133 ts_tablet_manager.cc:1905] T d728c7f604fb41178577fc079a9cef06 P 5b8af4b6cf364736a928ce119438e9a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:18:04.121510 14134 ts_tablet_manager.cc:1918] T d728c7f604fb41178577fc079a9cef06 P 408146e0c09b428b8f9dd510aeecbdc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:18:04.121665 14134 log.cc:1196] T d728c7f604fb41178577fc079a9cef06 P 408146e0c09b428b8f9dd510aeecbdc6: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/d728c7f604fb41178577fc079a9cef06
I20260809 20:18:04.121795 14133 ts_tablet_manager.cc:1918] T d728c7f604fb41178577fc079a9cef06 P 5b8af4b6cf364736a928ce119438e9a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:18:04.121991 14133 log.cc:1196] T d728c7f604fb41178577fc079a9cef06 P 5b8af4b6cf364736a928ce119438e9a3: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/d728c7f604fb41178577fc079a9cef06
I20260809 20:18:04.122035 14134 ts_tablet_manager.cc:1939] T d728c7f604fb41178577fc079a9cef06 P 408146e0c09b428b8f9dd510aeecbdc6: Deleting consensus metadata
I20260809 20:18:04.122586 14133 ts_tablet_manager.cc:1939] T d728c7f604fb41178577fc079a9cef06 P 5b8af4b6cf364736a928ce119438e9a3: Deleting consensus metadata
I20260809 20:18:04.123037 13238 catalog_manager.cc:4841] TS 408146e0c09b428b8f9dd510aeecbdc6 (127.5.61.3:33467): tablet d728c7f604fb41178577fc079a9cef06 (table test-table [id=bda0772a52384c71a06aaa92fa27a89d]) successfully deleted
I20260809 20:18:04.123791 13237 catalog_manager.cc:4841] TS 5b8af4b6cf364736a928ce119438e9a3 (127.5.61.1:37061): tablet d728c7f604fb41178577fc079a9cef06 (table test-table [id=bda0772a52384c71a06aaa92fa27a89d]) successfully deleted
I20260809 20:18:04.127357 14101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:04.102392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39380 (local address 127.5.61.1:37061)
0809 20:18:04.102622 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:04.102631 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:04.108033 (+ 5402us) server_negotiation.cc:407] Connection header received
0809 20:18:04.108335 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:04.108340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:04.108437 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:04.108507 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:04.110414 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:04.111565 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:04.115799 (+ 4234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:04.116508 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:04.116774 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:04.120391 (+ 3617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:04.120410 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:04.120430 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:04.120488 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:04.124620 (+ 4132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:04.125509 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:04.125516 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:04.125518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:04.125673 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:04.125996 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:04.125999 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:04.126001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:04.126339 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:04.126560 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:04.126873 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 20:18:04.127101 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 20:18:04.128026 13661 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:04.229836 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 13450
I20260809 20:18:04.251904 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 13681
I20260809 20:18:04.264842 14100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:04.252693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34608 (local address 127.5.61.2:38017)
0809 20:18:04.252891 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:04.252896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:04.252992 (+ 96us) server_negotiation.cc:407] Connection header received
0809 20:18:04.253199 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:04.253203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:04.253288 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:04.253374 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:04.254676 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:04.255405 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:04.256706 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:04.257085 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:04.257178 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:04.259730 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:04.259745 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:04.259760 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:04.259792 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:04.262428 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:04.263068 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:04.263074 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:04.263077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:04.263206 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:04.263542 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:04.263545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:04.263547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:04.263965 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:04.264197 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:04.264435 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 20:18:04.264592 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 20:18:04.265518 13826 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:04.388751 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 13681
I20260809 20:18:04.418179 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 13849
I20260809 20:18:04.432559 14103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:04.419046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42442 (local address 127.5.61.3:33467)
0809 20:18:04.419269 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:04.419277 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:04.419304 (+ 27us) server_negotiation.cc:407] Connection header received
0809 20:18:04.419540 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:04.419547 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:04.419699 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:04.419821 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:04.421504 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:04.422248 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:04.423490 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:04.424187 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:04.424331 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:04.426974 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:04.426989 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:04.427006 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:04.427043 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:04.429837 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:04.430441 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:04.430448 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:04.430451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:04.430573 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:04.430899 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:04.430902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:04.430904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:04.431238 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:04.431436 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:04.432024 (+ 588us) server_negotiation.cc:299] Negotiation successful
0809 20:18:04.432263 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 20:18:04.433300 13991 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:04.465678 14084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:04.465257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.0.0.1:55602)
0809 20:18:04.465442 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:04.465532 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 20:18:04.467017 14084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:04.466634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40647 (local address 127.0.0.1:55604)
0809 20:18:04.466824 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:04.466900 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 20:18:04.528756 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 13849
I20260809 20:18:04.549784 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 13094
I20260809 20:18:04.550863 13291 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:04.675179 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 13094
I20260809 20:18:04.702562 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 13215
I20260809 20:18:04.703696 13503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:04.819196 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 13215
2026-08-09T20:18:04Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9578 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:18:04 dist-test-slave-6k22 krb5kdc[14145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:18:04 dist-test-slave-6k22 krb5kdc[14145](info): set up 2 sockets
Aug 09 20:18:04 dist-test-slave-6k22 krb5kdc[14145](info): commencing operation
krb5kdc: starting...
W20260809 20:18:06.006284 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:18:06 dist-test-slave-6k22 krb5kdc[14145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:18:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:18:06Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.62.keytab.
Entry for principal oryx/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.62.keytab.
I20260809 20:18:06.255910 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:37709
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.62.keytab
--principal=oryx/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:37709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:18:06.581637 14158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:06.581907 14158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:06.582075 14158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:06.588858 14158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:18:06.588951 14158 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 20:18:06.589046 14158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:06.589172 14158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:06.589284 14158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:18:06.589406 14158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:18:06.598877 14158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:37709
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.62.keytab
--principal=oryx/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:37709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:06.600104 14158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:06.601442 14158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:06.614027 14166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:06.614095 14165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:06.614063 14168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:06.614843 14158 server_base.cc:1034] running on GCE node
I20260809 20:18:06.615694 14158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:06.617028 14158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:06.618271 14158 hybrid_clock.cc:648] HybridClock initialized: now 1786306686618205 us; error 31 us; skew 500 ppm
I20260809 20:18:06.618642 14158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:18:06 dist-test-slave-6k22 krb5kdc[14145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306686, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:18:06.626788 14158 init.cc:377] Logged in from keytab as oryx/127.5.61.62@KRBTEST.COM (short username oryx)
I20260809 20:18:06.628248 14158 webserver.cc:460] Webserver started at http://127.5.61.62:39039/ using document root <none> and password file <none>
I20260809 20:18:06.628931 14158 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:06.629065 14158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:06.629410 14158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:18:06.632473 14158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "cef8dad6d8e0499aa793c78c9ab3f50d"
format_stamp: "Formatted at 2026-08-09 20:18:06 on dist-test-slave-6k22"
server_key: "5d5afaf20efe509df65663f5518f0123"
server_key_iv: "870620fdde80c27734f54c92a1d08082"
server_key_version: "encryptionkey@0"
I20260809 20:18:06.633296 14158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "cef8dad6d8e0499aa793c78c9ab3f50d"
format_stamp: "Formatted at 2026-08-09 20:18:06 on dist-test-slave-6k22"
server_key: "5d5afaf20efe509df65663f5518f0123"
server_key_iv: "870620fdde80c27734f54c92a1d08082"
server_key_version: "encryptionkey@0"
I20260809 20:18:06.638273 14158 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 20:18:06.641633 14176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:06.642822 14158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 20:18:06.643011 14158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "cef8dad6d8e0499aa793c78c9ab3f50d"
format_stamp: "Formatted at 2026-08-09 20:18:06 on dist-test-slave-6k22"
server_key: "5d5afaf20efe509df65663f5518f0123"
server_key_iv: "870620fdde80c27734f54c92a1d08082"
server_key_version: "encryptionkey@0"
I20260809 20:18:06.643231 14158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:06.684512 14158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:06.689409 14158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:06.689749 14158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:06.712260 14158 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:37709
I20260809 20:18:06.712289 14240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:37709 every 8 connection(s)
I20260809 20:18:06.713577 14158 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:18:06.717155 14241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:06.720669 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 14158
I20260809 20:18:06.720884 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:18:06.721733 5364 external_mini_cluster.cc:1428] Setting key 7770d0d824d47ab7dc7c49df7ba52b09
I20260809 20:18:06.730595 14241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d: Bootstrap starting.
I20260809 20:18:06.734310 14241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:06.735699 14241 log.cc:824] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d: Log is configured to *not* fsync() on all Append() calls
Aug 09 20:18:06 dist-test-slave-6k22 krb5kdc[14145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM
I20260809 20:18:06.738692 14241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d: No bootstrap required, opened a new log
I20260809 20:18:06.745020 14244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:06.723869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49546 (local address 127.5.61.62:37709)
0809 20:18:06.724648 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:06.724659 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:06.724699 (+ 40us) server_negotiation.cc:407] Connection header received
0809 20:18:06.726282 (+ 1583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:06.726303 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:06.726695 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:06.727036 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:06.728946 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:06.730196 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:06.731941 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:06.732619 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:06.732748 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:06.737751 (+ 5003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:06.737758 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:06.737772 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:06.737805 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:06.740977 (+ 3172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:06.741760 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:06.741765 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:06.741770 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:06.741946 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:06.742451 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:06.742454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:06.742456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:06.742935 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:06.743166 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:06.743676 (+ 510us) server_negotiation.cc:299] Negotiation successful
0809 20:18:06.744066 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":382,"threads_started":1}
I20260809 20:18:06.747541 14241 raft_consensus.cc:348] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cef8dad6d8e0499aa793c78c9ab3f50d" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37709 } }
I20260809 20:18:06.748005 14241 raft_consensus.cc:374] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:06.748107 14241 raft_consensus.cc:729] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cef8dad6d8e0499aa793c78c9ab3f50d, State: Initialized, Role: FOLLOWER
I20260809 20:18:06.748709 14241 consensus_queue.cc:260] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cef8dad6d8e0499aa793c78c9ab3f50d" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37709 } }
I20260809 20:18:06.748960 14241 raft_consensus.cc:388] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:18:06.749089 14241 raft_consensus.cc:482] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:18:06.749280 14241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:06.751353 14241 raft_consensus.cc:504] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cef8dad6d8e0499aa793c78c9ab3f50d" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37709 } }
I20260809 20:18:06.751852 14241 leader_election.cc:302] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cef8dad6d8e0499aa793c78c9ab3f50d; no voters:
I20260809 20:18:06.752406 14241 leader_election.cc:288] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:18:06.752548 14246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:18:06.753309 14246 raft_consensus.cc:686] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d [term 1 LEADER]: Becoming Leader. State: Replica: cef8dad6d8e0499aa793c78c9ab3f50d, State: Running, Role: LEADER
I20260809 20:18:06.753823 14246 consensus_queue.cc:237] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cef8dad6d8e0499aa793c78c9ab3f50d" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37709 } }
I20260809 20:18:06.754451 14241 sys_catalog.cc:564] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:18:06.756752 14247 sys_catalog.cc:455] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cef8dad6d8e0499aa793c78c9ab3f50d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cef8dad6d8e0499aa793c78c9ab3f50d" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37709 } } }
I20260809 20:18:06.756886 14248 sys_catalog.cc:455] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d [sys.catalog]: SysCatalogTable state changed. Reason: New leader cef8dad6d8e0499aa793c78c9ab3f50d. Latest consensus state: current_term: 1 leader_uuid: "cef8dad6d8e0499aa793c78c9ab3f50d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cef8dad6d8e0499aa793c78c9ab3f50d" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37709 } } }
I20260809 20:18:06.757189 14247 sys_catalog.cc:458] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d [sys.catalog]: This master's current role is: LEADER
I20260809 20:18:06.757265 14248 sys_catalog.cc:458] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d [sys.catalog]: This master's current role is: LEADER
I20260809 20:18:06.758132 14253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:18:06.763621 14253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:18:06.772368 14253 catalog_manager.cc:1347] Generated new cluster ID: 7f75aeabe9a1480dbff9eaed51e6d58c
I20260809 20:18:06.772486 14253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:18:06.792507 14253 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:18:06.793823 14253 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:18:06.799293 14253 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d: Generated new TSK 0
I20260809 20:18:06.800164 14253 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.61.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.61.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.61.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.1.keytab.
Entry for principal oryx/127.5.61.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.1.keytab.
I20260809 20:18:06.876029 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:37709
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.1.keytab
--principal=oryx/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:18:07.206611 14271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:07.206871 14271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:07.207044 14271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:07.214099 14271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:07.214188 14271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:07.214370 14271 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 20:18:07.214493 14271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:18:07.223810 14271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.1.keytab
--principal=oryx/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.61.62:37709
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:07.224987 14271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:07.226306 14271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:07.239387 14276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:07.239495 14277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:07.239457 14279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:07.240583 14271 server_base.cc:1034] running on GCE node
I20260809 20:18:07.241194 14271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:07.242054 14271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:07.243314 14271 hybrid_clock.cc:648] HybridClock initialized: now 1786306687243256 us; error 35 us; skew 500 ppm
I20260809 20:18:07.243700 14271 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:18:07 dist-test-slave-6k22 krb5kdc[14145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306687, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:18:07.247246 14271 init.cc:377] Logged in from keytab as oryx/127.5.61.1@KRBTEST.COM (short username oryx)
I20260809 20:18:07.248675 14271 webserver.cc:460] Webserver started at http://127.5.61.1:32865/ using document root <none> and password file <none>
I20260809 20:18:07.249382 14271 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:07.249495 14271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:07.249853 14271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:18:07.252789 14271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "34cffa13487e4d1f87816716f16f3cb3"
format_stamp: "Formatted at 2026-08-09 20:18:07 on dist-test-slave-6k22"
server_key: "05a2b0611f09c00fa0ee0d43815f7f67"
server_key_iv: "7d8ba28af09f92ac04e9bbafb92cf1f4"
server_key_version: "encryptionkey@0"
I20260809 20:18:07.253607 14271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "34cffa13487e4d1f87816716f16f3cb3"
format_stamp: "Formatted at 2026-08-09 20:18:07 on dist-test-slave-6k22"
server_key: "05a2b0611f09c00fa0ee0d43815f7f67"
server_key_iv: "7d8ba28af09f92ac04e9bbafb92cf1f4"
server_key_version: "encryptionkey@0"
I20260809 20:18:07.258983 14271 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 20:18:07.262594 14287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:07.264021 14271 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:18:07.264179 14271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "34cffa13487e4d1f87816716f16f3cb3"
format_stamp: "Formatted at 2026-08-09 20:18:07 on dist-test-slave-6k22"
server_key: "05a2b0611f09c00fa0ee0d43815f7f67"
server_key_iv: "7d8ba28af09f92ac04e9bbafb92cf1f4"
server_key_version: "encryptionkey@0"
I20260809 20:18:07.264393 14271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:07.288705 14271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:07.293140 14271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:07.293486 14271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:07.296087 14271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:18:07.296227 14271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:07.296392 14271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:18:07.296475 14271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 20:18:07 dist-test-slave-6k22 krb5kdc[14145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306687, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM
I20260809 20:18:07.318825 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:07.296975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:42875 (local address 127.5.61.62:37709)
0809 20:18:07.297532 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:07.297539 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:07.298875 (+ 1336us) server_negotiation.cc:407] Connection header received
0809 20:18:07.300930 (+ 2055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:07.300936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:07.301019 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:07.301181 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:07.303167 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:07.304307 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:07.306691 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:07.307144 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:07.307288 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:07.311900 (+ 4612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:07.311907 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:07.311915 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:07.311955 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:07.315439 (+ 3484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:07.316341 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:07.316349 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:07.316352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:07.316490 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:07.316938 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:07.316943 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:07.316948 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:07.317222 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:07.317397 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:07.318200 (+ 803us) server_negotiation.cc:299] Negotiation successful
0809 20:18:07.318473 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":304,"threads_started":1}
I20260809 20:18:07.320035 14299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:07.297563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:37709 (local address 127.5.61.1:42875)
0809 20:18:07.298500 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:07.298557 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:07.300608 (+ 2051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:07.301397 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:07.301411 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:07.302067 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:07.302925 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:07.302954 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:07.304489 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:07.304496 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:07.306526 (+ 2030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:07.306540 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:07.307381 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:07.307391 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:07.307678 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:07.308403 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:07.308442 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:07.311525 (+ 3083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:07.315608 (+ 4083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:07.315618 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:07.315635 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:07.316203 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:07.316607 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:07.316612 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:07.316616 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:07.316805 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:07.317360 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:07.317368 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:07.317689 (+ 321us) client_negotiation.cc:769] Sending connection context
0809 20:18:07.317951 (+ 262us) client_negotiation.cc:240] Negotiation successful
0809 20:18:07.318526 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":397,"threads_started":1}
I20260809 20:18:07.348955 14371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:07.331404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:37709 (local address 127.5.61.1:60215)
0809 20:18:07.332578 (+ 1174us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:07.332609 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:07.332797 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:07.333193 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:07.333198 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:07.333969 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:07.334526 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:07.334540 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:07.335947 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:07.335952 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:07.337931 (+ 1979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:07.337953 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:07.339002 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:07.339009 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:07.339175 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:07.340003 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:07.340030 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:07.341115 (+ 1085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:07.345663 (+ 4548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:07.345670 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:07.345676 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:07.346271 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:07.346746 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:07.346751 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:07.346755 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:07.346887 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:07.347493 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:07.347498 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:07.347794 (+ 296us) client_negotiation.cc:769] Sending connection context
0809 20:18:07.347986 (+ 192us) client_negotiation.cc:240] Negotiation successful
0809 20:18:07.348333 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":985,"thread_start_us":736,"threads_started":1}
I20260809 20:18:07.349829 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:07.331380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:60215 (local address 127.5.61.62:37709)
0809 20:18:07.331625 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:07.331633 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:07.332711 (+ 1078us) server_negotiation.cc:407] Connection header received
0809 20:18:07.332963 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:07.332967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:07.333047 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:07.333164 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:07.334691 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:07.335742 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:07.338135 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:07.338753 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:07.338906 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:07.341329 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:07.341336 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:07.341341 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:07.341384 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:07.345517 (+ 4133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:07.346425 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:07.346430 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:07.346434 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:07.346633 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:07.346999 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:07.347025 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:07.347028 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:07.347368 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:07.347525 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:07.349438 (+ 1913us) server_negotiation.cc:299] Negotiation successful
0809 20:18:07.349599 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 20:18:07.352787 14271 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:40493
I20260809 20:18:07.352844 14419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:40493 every 8 connection(s)
I20260809 20:18:07.353981 14271 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:18:07.356128 14420 heartbeater.cc:347] Connected to a master server at 127.5.61.62:37709
I20260809 20:18:07.356366 14420 heartbeater.cc:464] Registering TS with master...
I20260809 20:18:07.357048 14420 heartbeater.cc:511] Master 127.5.61.62:37709 requested a full tablet report, sending...
I20260809 20:18:07.359048 14195 ts_manager.cc:194] Registered new tserver with Master: 34cffa13487e4d1f87816716f16f3cb3 (127.5.61.1:40493)
I20260809 20:18:07.360631 14195 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.61.1@KRBTEST.COM'} at 127.5.61.1:42875
I20260809 20:18:07.363025 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 14271
I20260809 20:18:07.363224 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:18:07.363986 5364 external_mini_cluster.cc:1428] Setting key 2f889a4b3523ea258ac42769ab75554d
WARNING: no policy specified for oryx/127.5.61.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.61.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.61.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.2.keytab.
Entry for principal oryx/127.5.61.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.2.keytab.
I20260809 20:18:07.419265 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:37709
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.2.keytab
--principal=oryx/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:18:07.752122 14424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:07.752403 14424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:07.752502 14424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:07.760046 14424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:07.760193 14424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:07.760303 14424 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 20:18:07.760381 14424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:18:07.769836 14424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.2.keytab
--principal=oryx/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.61.62:37709
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:07.770924 14424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:07.772243 14424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:07.785197 14430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:07.785231 14429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:07.785836 14432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:07.786268 14424 server_base.cc:1034] running on GCE node
I20260809 20:18:07.786938 14424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:07.787910 14424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:07.789157 14424 hybrid_clock.cc:648] HybridClock initialized: now 1786306687789098 us; error 56 us; skew 500 ppm
I20260809 20:18:07.789451 14424 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:18:07 dist-test-slave-6k22 krb5kdc[14145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306687, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:18:07.792721 14424 init.cc:377] Logged in from keytab as oryx/127.5.61.2@KRBTEST.COM (short username oryx)
I20260809 20:18:07.794044 14424 webserver.cc:460] Webserver started at http://127.5.61.2:37223/ using document root <none> and password file <none>
I20260809 20:18:07.794696 14424 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:07.794780 14424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:07.795064 14424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:18:07.797973 14424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "442694a685c043e99fc1ad2e38b94b21"
format_stamp: "Formatted at 2026-08-09 20:18:07 on dist-test-slave-6k22"
server_key: "68a2b7c4a80bcb2ac9edbfaee4b10214"
server_key_iv: "ff3abb5923b1fc5e8203067257a4c7e5"
server_key_version: "encryptionkey@0"
I20260809 20:18:07.798729 14424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "442694a685c043e99fc1ad2e38b94b21"
format_stamp: "Formatted at 2026-08-09 20:18:07 on dist-test-slave-6k22"
server_key: "68a2b7c4a80bcb2ac9edbfaee4b10214"
server_key_iv: "ff3abb5923b1fc5e8203067257a4c7e5"
server_key_version: "encryptionkey@0"
I20260809 20:18:07.803614 14424 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 20:18:07.807025 14441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:07.808421 14424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 20:18:07.808570 14424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "442694a685c043e99fc1ad2e38b94b21"
format_stamp: "Formatted at 2026-08-09 20:18:07 on dist-test-slave-6k22"
server_key: "68a2b7c4a80bcb2ac9edbfaee4b10214"
server_key_iv: "ff3abb5923b1fc5e8203067257a4c7e5"
server_key_version: "encryptionkey@0"
I20260809 20:18:07.808755 14424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:07.827075 14424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:07.831518 14424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:07.831813 14424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:07.834576 14424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:18:07.834686 14424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:07.834915 14424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:18:07.835013 14424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 20:18:07 dist-test-slave-6k22 krb5kdc[14145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306687, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM
I20260809 20:18:07.858155 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:07.835077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:54407 (local address 127.5.61.62:37709)
0809 20:18:07.835280 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:07.835287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:07.836715 (+ 1428us) server_negotiation.cc:407] Connection header received
0809 20:18:07.838752 (+ 2037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:07.838757 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:07.838851 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:07.838968 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:07.841095 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:07.842434 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:07.845227 (+ 2793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:07.845816 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:07.845952 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:07.850903 (+ 4951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:07.850909 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:07.850912 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:07.850942 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:07.854173 (+ 3231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:07.855197 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:07.855202 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:07.855206 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:07.855335 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:07.855966 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:07.855973 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:07.855978 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:07.856401 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:07.856567 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:07.857298 (+ 731us) server_negotiation.cc:299] Negotiation successful
0809 20:18:07.857605 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 20:18:07.858620 14450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:07.835639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:37709 (local address 127.5.61.2:54407)
0809 20:18:07.836523 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:07.836578 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:07.838462 (+ 1884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:07.839222 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:07.839231 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:07.839989 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:07.840827 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:07.840854 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:07.842754 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:07.842761 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:07.845021 (+ 2260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:07.845040 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:07.846070 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:07.846077 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:07.846417 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:07.847217 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:07.847270 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:07.850646 (+ 3376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:07.854319 (+ 3673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:07.854346 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:07.854359 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:07.855058 (+ 699us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:07.855465 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:07.855473 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:07.855478 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:07.855813 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:07.856550 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:07.856557 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:07.856858 (+ 301us) client_negotiation.cc:769] Sending connection context
0809 20:18:07.857102 (+ 244us) client_negotiation.cc:240] Negotiation successful
0809 20:18:07.857457 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":396,"threads_started":1}
I20260809 20:18:07.882737 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:07.865023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:37313 (local address 127.5.61.62:37709)
0809 20:18:07.865196 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:07.865204 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:07.866146 (+ 942us) server_negotiation.cc:407] Connection header received
0809 20:18:07.866476 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:07.866481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:07.866580 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:07.866688 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:07.868063 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:07.869237 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:07.871375 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:07.872085 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:07.872221 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:07.875809 (+ 3588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:07.875817 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:07.875823 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:07.875860 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:07.879344 (+ 3484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:07.880368 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:07.880374 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:07.880377 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:07.880533 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:07.880992 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:07.880997 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:07.881000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:07.881468 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:07.881635 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:07.882028 (+ 393us) server_negotiation.cc:299] Negotiation successful
0809 20:18:07.882311 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 20:18:07.882792 14524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:07.864992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:37709 (local address 127.5.61.2:37313)
0809 20:18:07.866048 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:07.866066 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:07.866277 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:07.866749 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:07.866758 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:07.867497 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:07.867923 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:07.867935 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:07.869368 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:07.869374 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:07.871199 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:07.871219 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:07.872461 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:07.872470 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:07.872749 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:07.873883 (+ 1134us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:07.873908 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:07.875517 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:07.879580 (+ 4063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:07.879598 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:07.879603 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:07.880183 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:07.880629 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:07.880634 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:07.880637 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:07.880819 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:07.881599 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:07.881602 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:07.881757 (+ 155us) client_negotiation.cc:769] Sending connection context
0809 20:18:07.881873 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 20:18:07.882090 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":906,"thread_start_us":3640,"threads_started":1}
I20260809 20:18:07.890498 14424 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:44787
I20260809 20:18:07.890565 14577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:44787 every 8 connection(s)
I20260809 20:18:07.891712 14424 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:18:07.893817 14578 heartbeater.cc:347] Connected to a master server at 127.5.61.62:37709
I20260809 20:18:07.894058 14578 heartbeater.cc:464] Registering TS with master...
I20260809 20:18:07.894119 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 14424
I20260809 20:18:07.894291 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:18:07.894872 14578 heartbeater.cc:511] Master 127.5.61.62:37709 requested a full tablet report, sending...
I20260809 20:18:07.894974 5364 external_mini_cluster.cc:1428] Setting key 42889dee8221e100e3c79584ce9b283e
I20260809 20:18:07.896310 14195 ts_manager.cc:194] Registered new tserver with Master: 442694a685c043e99fc1ad2e38b94b21 (127.5.61.2:44787)
I20260809 20:18:07.897289 14195 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.61.2@KRBTEST.COM'} at 127.5.61.2:54407
WARNING: no policy specified for oryx/127.5.61.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.61.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.61.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.3.keytab.
Entry for principal oryx/127.5.61.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.3.keytab.
I20260809 20:18:07.951244 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:37709
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.3.keytab
--principal=oryx/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:18:08.280378 14582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:08.280642 14582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:08.280822 14582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:08.287830 14582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:08.287930 14582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:08.288111 14582 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 20:18:08.288260 14582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:18:08.297756 14582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.3.keytab
--principal=oryx/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.61.62:37709
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:08.298992 14582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:08.300300 14582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:08.313506 14588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:08.313535 14591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:08.313658 14589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:08.314527 14582 server_base.cc:1034] running on GCE node
I20260809 20:18:08.315047 14582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:08.315909 14582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:08.317157 14582 hybrid_clock.cc:648] HybridClock initialized: now 1786306688317097 us; error 39 us; skew 500 ppm
I20260809 20:18:08.317463 14582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:18:08 dist-test-slave-6k22 krb5kdc[14145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306688, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:18:08.320672 14582 init.cc:377] Logged in from keytab as oryx/127.5.61.3@KRBTEST.COM (short username oryx)
I20260809 20:18:08.322086 14582 webserver.cc:460] Webserver started at http://127.5.61.3:43129/ using document root <none> and password file <none>
I20260809 20:18:08.322760 14582 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:08.322914 14582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:08.323277 14582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:18:08.326169 14582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "54ee8649ea6247caa5df66298ed56767"
format_stamp: "Formatted at 2026-08-09 20:18:08 on dist-test-slave-6k22"
server_key: "2f10e68a303f2338ceb5b716338bb9fd"
server_key_iv: "b77ee4efcde4fcd372892a23b5a823ef"
server_key_version: "encryptionkey@0"
I20260809 20:18:08.326938 14582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "54ee8649ea6247caa5df66298ed56767"
format_stamp: "Formatted at 2026-08-09 20:18:08 on dist-test-slave-6k22"
server_key: "2f10e68a303f2338ceb5b716338bb9fd"
server_key_iv: "b77ee4efcde4fcd372892a23b5a823ef"
server_key_version: "encryptionkey@0"
I20260809 20:18:08.332018 14582 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 20:18:08.335323 14600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:08.336628 14582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:18:08.336800 14582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "54ee8649ea6247caa5df66298ed56767"
format_stamp: "Formatted at 2026-08-09 20:18:08 on dist-test-slave-6k22"
server_key: "2f10e68a303f2338ceb5b716338bb9fd"
server_key_iv: "b77ee4efcde4fcd372892a23b5a823ef"
server_key_version: "encryptionkey@0"
I20260809 20:18:08.337013 14582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:08.348610 14582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:08.352315 14582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:08.352635 14582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:08.355512 14582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:18:08.355736 14582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:08.355926 14582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:18:08.356047 14582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:08.363226 14420 heartbeater.cc:503] Master 127.5.61.62:37709 was elected leader, sending a full tablet report...
Aug 09 20:18:08 dist-test-slave-6k22 krb5kdc[14145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306688, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM
I20260809 20:18:08.380333 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.356139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:55081 (local address 127.5.61.62:37709)
0809 20:18:08.356348 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:08.356358 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:08.357625 (+ 1267us) server_negotiation.cc:407] Connection header received
0809 20:18:08.360501 (+ 2876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:08.360520 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:08.360603 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:08.360724 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:08.362544 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.363754 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.367157 (+ 3403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.367941 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.368088 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.373460 (+ 5372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:08.373468 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:08.373475 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:08.373524 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:08.376987 (+ 3463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:08.377775 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:08.377781 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:08.377785 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:08.377927 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:08.378421 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:08.378428 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:08.378431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:08.378744 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:08.378903 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:08.379675 (+ 772us) server_negotiation.cc:299] Negotiation successful
0809 20:18:08.379937 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 20:18:08.381716 14611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.356706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:37709 (local address 127.5.61.3:55081)
0809 20:18:08.357465 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:08.357502 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:08.360151 (+ 2649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:08.361039 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:08.361050 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:08.361689 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:08.362357 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:08.362376 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.363990 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.363997 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:08.366895 (+ 2898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:08.366916 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.368264 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.368273 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:08.368739 (+ 466us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.369624 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:08.369657 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:08.373168 (+ 3511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:08.377177 (+ 4009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:08.377190 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:08.377203 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:08.377639 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:08.378068 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:08.378076 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:08.378080 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:08.378283 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:08.378888 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:08.378897 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:08.379266 (+ 369us) client_negotiation.cc:769] Sending connection context
0809 20:18:08.379553 (+ 287us) client_negotiation.cc:240] Negotiation successful
0809 20:18:08.380208 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":331,"threads_started":1}
I20260809 20:18:08.402812 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.387920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:49625 (local address 127.5.61.62:37709)
0809 20:18:08.388325 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:08.388332 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:08.388800 (+ 468us) server_negotiation.cc:407] Connection header received
0809 20:18:08.388996 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:08.389001 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:08.389070 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:08.389169 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:08.390082 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.390785 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.392500 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.393142 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.393305 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.396081 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:08.396085 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:08.396088 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:08.396117 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:08.399789 (+ 3672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:08.400802 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:08.400809 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:08.400814 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:08.400954 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:08.401338 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:08.401344 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:08.401347 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:08.401618 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:08.401774 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:08.402208 (+ 434us) server_negotiation.cc:299] Negotiation successful
0809 20:18:08.402443 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257}
I20260809 20:18:08.402905 14676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.388015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:37709 (local address 127.5.61.3:49625)
0809 20:18:08.388695 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:08.388713 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:08.388868 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:08.389191 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:08.389196 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:08.389664 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:08.389942 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:08.389951 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.390979 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.390985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:08.392357 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:08.392369 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.393527 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.393557 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:08.393856 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.394641 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:08.394673 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:08.395885 (+ 1212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:08.399933 (+ 4048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:08.399938 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:08.399942 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:08.400655 (+ 713us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:08.401069 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:08.401074 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:08.401077 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:08.401214 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:08.401747 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:08.401752 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:08.401933 (+ 181us) client_negotiation.cc:769] Sending connection context
0809 20:18:08.402075 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 20:18:08.402426 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":439,"threads_started":1}
I20260809 20:18:08.415256 14582 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:41589
I20260809 20:18:08.415299 14734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:41589 every 8 connection(s)
I20260809 20:18:08.416545 14582 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:18:08.418628 14735 heartbeater.cc:347] Connected to a master server at 127.5.61.62:37709
I20260809 20:18:08.418824 14735 heartbeater.cc:464] Registering TS with master...
I20260809 20:18:08.419437 14735 heartbeater.cc:511] Master 127.5.61.62:37709 requested a full tablet report, sending...
I20260809 20:18:08.420814 14195 ts_manager.cc:194] Registered new tserver with Master: 54ee8649ea6247caa5df66298ed56767 (127.5.61.3:41589)
I20260809 20:18:08.421686 14195 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.61.3@KRBTEST.COM'} at 127.5.61.3:55081
I20260809 20:18:08.424194 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 14582
I20260809 20:18:08.424379 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:18:08.424822 5364 external_mini_cluster.cc:1428] Setting key 053acca01a150912e49f9d3c19a193d7
I20260809 20:18:08.427680 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 20:18:08 dist-test-slave-6k22 krb5kdc[14145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306688, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 20:18:08 dist-test-slave-6k22 krb5kdc[14145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:08 dist-test-slave-6k22 krb5kdc[14145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 20:18:08.481627 14298 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:08.471299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51264 (local address 127.5.61.62:37709)
0809 20:18:08.471496 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:08.471500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:08.471905 (+ 405us) server_negotiation.cc:407] Connection header received
0809 20:18:08.472055 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:08.472059 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:08.472124 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:08.472220 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:08.473478 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.474189 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.475415 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.475906 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.476004 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.481256 (+ 5252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51264: BlockingRecv error: recv got EOF from 127.0.0.1:51264 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 09 20:18:08 dist-test-slave-6k22 krb5kdc[14145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306688, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM
I20260809 20:18:08.498029 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.484557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51270 (local address 127.5.61.62:37709)
0809 20:18:08.484763 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:08.484770 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:08.485086 (+ 316us) server_negotiation.cc:407] Connection header received
0809 20:18:08.485311 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:08.485320 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:08.485428 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:08.485550 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:08.486611 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.487353 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.488708 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.489105 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.489298 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.492835 (+ 3537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:08.492840 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:08.492843 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:08.492879 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:08.495300 (+ 2421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:08.496063 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:08.496071 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:08.496074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:08.496200 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:08.496601 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:08.496608 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:08.496611 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:08.496886 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:08.497209 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:08.497502 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 20:18:08.497730 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 20:18:08.501842 14195 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51270:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:18:08.508760 14195 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:18:08.544359 14755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.521440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:40493 (local address 127.0.0.1:50810)
0809 20:18:08.533009 (+ 11569us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:08.533063 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:08.533303 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:08.534739 (+ 1436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:08.534748 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:08.534801 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:08.535376 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:08.535392 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.538164 (+ 2772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.538173 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:08.541269 (+ 3096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:08.541285 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.543307 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.543311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:08.543533 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.543622 (+ 89us) client_negotiation.cc:769] Sending connection context
0809 20:18:08.543746 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 20:18:08.543964 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11186,"mutex_wait_us":2,"thread_start_us":12850,"threads_started":1}
I20260809 20:18:08.545270 14754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.521533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50810 (local address 127.5.61.1:40493)
0809 20:18:08.534045 (+ 12512us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:08.534057 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:08.534094 (+ 37us) server_negotiation.cc:407] Connection header received
0809 20:18:08.534215 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:08.534228 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:08.534519 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:08.535677 (+ 1158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:08.535982 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.537967 (+ 1985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.541475 (+ 3508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.543051 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.543855 (+ 804us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.544066 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:08.544154 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 20:18:08.544321 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12285,"thread_start_us":12117,"threads_started":1}
I20260809 20:18:08.546433 14753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.521816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:44787 (local address 127.0.0.1:41986)
0809 20:18:08.534561 (+ 12745us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:08.534574 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:08.534725 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:08.535733 (+ 1008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:08.535752 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:08.535786 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:08.537777 (+ 1991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:08.537796 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.541804 (+ 4008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.541810 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:08.543618 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:08.543634 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.545559 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.545564 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:08.545792 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.545827 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 20:18:08.545914 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 20:18:08.546045 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12639,"mutex_wait_us":2,"thread_start_us":10629,"threads_started":1}
I20260809 20:18:08.546521 14756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.522302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41986 (local address 127.5.61.2:44787)
0809 20:18:08.532404 (+ 10102us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:08.532417 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:08.534876 (+ 2459us) server_negotiation.cc:407] Connection header received
0809 20:18:08.534991 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:08.534999 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:08.535230 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:08.535437 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:08.538026 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.541593 (+ 3567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.543776 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.545111 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.545227 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.545445 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:08.546088 (+ 643us) server_negotiation.cc:299] Negotiation successful
0809 20:18:08.546267 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9855,"thread_start_us":9699,"threads_started":1}
I20260809 20:18:08.549124 14345 tablet_service.cc:1467] Processing CreateTablet for tablet 1a42bfe6fd2d4a118730585bb0e46b23 (DEFAULT_TABLE table=test-table [id=4828d6c72d0d4a6fae75a8c97f9b1579]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:18:08.550738 14345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a42bfe6fd2d4a118730585bb0e46b23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:08.550350 14498 tablet_service.cc:1467] Processing CreateTablet for tablet 1a42bfe6fd2d4a118730585bb0e46b23 (DEFAULT_TABLE table=test-table [id=4828d6c72d0d4a6fae75a8c97f9b1579]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:18:08.550763 14758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.534695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:41589 (local address 127.0.0.1:50730)
0809 20:18:08.541269 (+ 6574us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:08.541287 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:08.541408 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:08.542790 (+ 1382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:08.542795 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:08.542822 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:08.543292 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:08.543304 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.545288 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.545296 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:08.547312 (+ 2016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:08.547325 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.549134 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.549143 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:08.549526 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.549558 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 20:18:08.549653 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 20:18:08.550440 (+ 787us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6443,"thread_start_us":6784,"threads_started":1}
I20260809 20:18:08.551301 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.535120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50730 (local address 127.5.61.3:41589)
0809 20:18:08.542200 (+ 7080us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:08.542207 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:08.542239 (+ 32us) server_negotiation.cc:407] Connection header received
0809 20:18:08.542327 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:08.542334 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:08.542560 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:08.542770 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:08.543544 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.545118 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.547477 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.548905 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.549039 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.549243 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:08.550145 (+ 902us) server_negotiation.cc:299] Negotiation successful
0809 20:18:08.550989 (+ 844us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6921,"thread_start_us":6671,"threads_started":1}
I20260809 20:18:08.551548 14498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a42bfe6fd2d4a118730585bb0e46b23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:08.557701 14656 tablet_service.cc:1467] Processing CreateTablet for tablet 1a42bfe6fd2d4a118730585bb0e46b23 (DEFAULT_TABLE table=test-table [id=4828d6c72d0d4a6fae75a8c97f9b1579]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:18:08.558845 14656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a42bfe6fd2d4a118730585bb0e46b23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:08.564548 14762 tablet_bootstrap.cc:492] T 1a42bfe6fd2d4a118730585bb0e46b23 P 34cffa13487e4d1f87816716f16f3cb3: Bootstrap starting.
I20260809 20:18:08.568058 14763 tablet_bootstrap.cc:492] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21: Bootstrap starting.
I20260809 20:18:08.568224 14762 tablet_bootstrap.cc:654] T 1a42bfe6fd2d4a118730585bb0e46b23 P 34cffa13487e4d1f87816716f16f3cb3: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:08.569545 14762 log.cc:824] T 1a42bfe6fd2d4a118730585bb0e46b23 P 34cffa13487e4d1f87816716f16f3cb3: Log is configured to *not* fsync() on all Append() calls
I20260809 20:18:08.571425 14763 tablet_bootstrap.cc:654] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:08.572391 14765 tablet_bootstrap.cc:492] T 1a42bfe6fd2d4a118730585bb0e46b23 P 54ee8649ea6247caa5df66298ed56767: Bootstrap starting.
I20260809 20:18:08.572789 14763 log.cc:824] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21: Log is configured to *not* fsync() on all Append() calls
I20260809 20:18:08.573694 14762 tablet_bootstrap.cc:492] T 1a42bfe6fd2d4a118730585bb0e46b23 P 34cffa13487e4d1f87816716f16f3cb3: No bootstrap required, opened a new log
I20260809 20:18:08.573993 14762 ts_tablet_manager.cc:1397] T 1a42bfe6fd2d4a118730585bb0e46b23 P 34cffa13487e4d1f87816716f16f3cb3: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260809 20:18:08.575896 14763 tablet_bootstrap.cc:492] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21: No bootstrap required, opened a new log
I20260809 20:18:08.575897 14765 tablet_bootstrap.cc:654] T 1a42bfe6fd2d4a118730585bb0e46b23 P 54ee8649ea6247caa5df66298ed56767: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:08.576169 14763 ts_tablet_manager.cc:1397] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 20:18:08.577124 14765 log.cc:824] T 1a42bfe6fd2d4a118730585bb0e46b23 P 54ee8649ea6247caa5df66298ed56767: Log is configured to *not* fsync() on all Append() calls
I20260809 20:18:08.580137 14765 tablet_bootstrap.cc:492] T 1a42bfe6fd2d4a118730585bb0e46b23 P 54ee8649ea6247caa5df66298ed56767: No bootstrap required, opened a new log
I20260809 20:18:08.580389 14765 ts_tablet_manager.cc:1397] T 1a42bfe6fd2d4a118730585bb0e46b23 P 54ee8649ea6247caa5df66298ed56767: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 20:18:08.582054 14762 raft_consensus.cc:348] T 1a42bfe6fd2d4a118730585bb0e46b23 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.582692 14762 raft_consensus.cc:374] T 1a42bfe6fd2d4a118730585bb0e46b23 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:08.582849 14762 raft_consensus.cc:729] T 1a42bfe6fd2d4a118730585bb0e46b23 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34cffa13487e4d1f87816716f16f3cb3, State: Initialized, Role: FOLLOWER
I20260809 20:18:08.583550 14762 consensus_queue.cc:260] T 1a42bfe6fd2d4a118730585bb0e46b23 P 34cffa13487e4d1f87816716f16f3cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.585258 14762 ts_tablet_manager.cc:1428] T 1a42bfe6fd2d4a118730585bb0e46b23 P 34cffa13487e4d1f87816716f16f3cb3: Time spent starting tablet: real 0.011s user 0.005s sys 0.005s
I20260809 20:18:08.588030 14763 raft_consensus.cc:348] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.588461 14763 raft_consensus.cc:374] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:08.588553 14763 raft_consensus.cc:729] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442694a685c043e99fc1ad2e38b94b21, State: Initialized, Role: FOLLOWER
I20260809 20:18:08.589134 14763 consensus_queue.cc:260] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.590220 14578 heartbeater.cc:503] Master 127.5.61.62:37709 was elected leader, sending a full tablet report...
I20260809 20:18:08.590845 14763 ts_tablet_manager.cc:1428] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260809 20:18:08.590927 14765 raft_consensus.cc:348] T 1a42bfe6fd2d4a118730585bb0e46b23 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.591259 14765 raft_consensus.cc:374] T 1a42bfe6fd2d4a118730585bb0e46b23 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:08.591377 14765 raft_consensus.cc:729] T 1a42bfe6fd2d4a118730585bb0e46b23 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54ee8649ea6247caa5df66298ed56767, State: Initialized, Role: FOLLOWER
I20260809 20:18:08.592022 14765 consensus_queue.cc:260] T 1a42bfe6fd2d4a118730585bb0e46b23 P 54ee8649ea6247caa5df66298ed56767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.593137 14735 heartbeater.cc:503] Master 127.5.61.62:37709 was elected leader, sending a full tablet report...
I20260809 20:18:08.593675 14765 ts_tablet_manager.cc:1428] T 1a42bfe6fd2d4a118730585bb0e46b23 P 54ee8649ea6247caa5df66298ed56767: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
W20260809 20:18:08.606567 14421 tablet.cc:2369] T 1a42bfe6fd2d4a118730585bb0e46b23 P 34cffa13487e4d1f87816716f16f3cb3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 20:18:08.644661 14579 tablet.cc:2369] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:18:08.648622 14769 raft_consensus.cc:482] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:18:08.648912 14769 raft_consensus.cc:504] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.650485 14769 leader_election.cc:288] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34cffa13487e4d1f87816716f16f3cb3 (127.5.61.1:40493), 54ee8649ea6247caa5df66298ed56767 (127.5.61.3:41589)
I20260809 20:18:08.658706 14754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.651074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:50135 (local address 127.5.61.1:40493)
0809 20:18:08.651275 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:08.651279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:08.651929 (+ 650us) server_negotiation.cc:407] Connection header received
0809 20:18:08.652119 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:08.652124 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:08.652225 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:08.652343 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:08.653055 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.654034 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.656623 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.657569 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.657677 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.657753 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:08.658170 (+ 417us) server_negotiation.cc:299] Negotiation successful
0809 20:18:08.658388 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 20:18:08.658766 14772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.650958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:40493 (local address 127.5.61.2:50135)
0809 20:18:08.651772 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:08.651792 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:08.651949 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:08.652438 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:08.652443 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:08.652482 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:08.652879 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:08.652892 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.654171 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.654176 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:08.656463 (+ 2287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:08.656476 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.657810 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.657817 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:08.657991 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.658025 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 20:18:08.658120 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 20:18:08.658261 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"mutex_wait_us":90,"thread_start_us":433,"threads_started":1}
I20260809 20:18:08.659032 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.652136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:40043 (local address 127.5.61.3:41589)
0809 20:18:08.652345 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:08.652351 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:08.652671 (+ 320us) server_negotiation.cc:407] Connection header received
0809 20:18:08.652831 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:08.652838 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:08.652917 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:08.653006 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:08.653685 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.654542 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.657072 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.658002 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.658135 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.658233 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:08.658643 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 20:18:08.658794 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 20:18:08.659196 14774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.651862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:41589 (local address 127.5.61.2:40043)
0809 20:18:08.652553 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:08.652571 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:08.652696 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:08.653051 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:08.653055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:08.653079 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:08.653505 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:08.653517 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.654725 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.654730 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:08.656910 (+ 2180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:08.656924 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.658262 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.658269 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:08.658478 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.658513 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 20:18:08.658615 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 20:18:08.658761 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"mutex_wait_us":20,"spinlock_wait_cycles":23936,"thread_start_us":383,"threads_started":1}
I20260809 20:18:08.660466 14373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a42bfe6fd2d4a118730585bb0e46b23" candidate_uuid: "442694a685c043e99fc1ad2e38b94b21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34cffa13487e4d1f87816716f16f3cb3" is_pre_election: true
I20260809 20:18:08.660461 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a42bfe6fd2d4a118730585bb0e46b23" candidate_uuid: "442694a685c043e99fc1ad2e38b94b21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54ee8649ea6247caa5df66298ed56767" is_pre_election: true
I20260809 20:18:08.661113 14373 raft_consensus.cc:2454] T 1a42bfe6fd2d4a118730585bb0e46b23 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442694a685c043e99fc1ad2e38b94b21 in term 0.
I20260809 20:18:08.661135 14688 raft_consensus.cc:2454] T 1a42bfe6fd2d4a118730585bb0e46b23 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 442694a685c043e99fc1ad2e38b94b21 in term 0.
I20260809 20:18:08.661816 14445 leader_election.cc:302] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34cffa13487e4d1f87816716f16f3cb3, 442694a685c043e99fc1ad2e38b94b21; no voters:
I20260809 20:18:08.662226 14769 raft_consensus.cc:2781] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:18:08.662357 14769 raft_consensus.cc:482] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:18:08.662521 14769 raft_consensus.cc:3037] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:08.664765 14769 raft_consensus.cc:504] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.665313 14769 leader_election.cc:288] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21 [CANDIDATE]: Term 1 election: Requested vote from peers 34cffa13487e4d1f87816716f16f3cb3 (127.5.61.1:40493), 54ee8649ea6247caa5df66298ed56767 (127.5.61.3:41589)
I20260809 20:18:08.665900 14373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a42bfe6fd2d4a118730585bb0e46b23" candidate_uuid: "442694a685c043e99fc1ad2e38b94b21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34cffa13487e4d1f87816716f16f3cb3"
I20260809 20:18:08.665946 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a42bfe6fd2d4a118730585bb0e46b23" candidate_uuid: "442694a685c043e99fc1ad2e38b94b21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54ee8649ea6247caa5df66298ed56767"
I20260809 20:18:08.666126 14688 raft_consensus.cc:3037] T 1a42bfe6fd2d4a118730585bb0e46b23 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:08.666126 14373 raft_consensus.cc:3037] T 1a42bfe6fd2d4a118730585bb0e46b23 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Advancing to term 1
W20260809 20:18:08.668190 14736 tablet.cc:2369] T 1a42bfe6fd2d4a118730585bb0e46b23 P 54ee8649ea6247caa5df66298ed56767: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:18:08.669525 14373 raft_consensus.cc:2454] T 1a42bfe6fd2d4a118730585bb0e46b23 P 34cffa13487e4d1f87816716f16f3cb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442694a685c043e99fc1ad2e38b94b21 in term 1.
I20260809 20:18:08.669524 14688 raft_consensus.cc:2454] T 1a42bfe6fd2d4a118730585bb0e46b23 P 54ee8649ea6247caa5df66298ed56767 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 442694a685c043e99fc1ad2e38b94b21 in term 1.
I20260809 20:18:08.669909 14445 leader_election.cc:302] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442694a685c043e99fc1ad2e38b94b21, 54ee8649ea6247caa5df66298ed56767; no voters:
I20260809 20:18:08.670177 14769 raft_consensus.cc:2781] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:18:08.670636 14769 raft_consensus.cc:686] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21 [term 1 LEADER]: Becoming Leader. State: Replica: 442694a685c043e99fc1ad2e38b94b21, State: Running, Role: LEADER
I20260809 20:18:08.671092 14769 consensus_queue.cc:237] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.676097 14195 catalog_manager.cc:5495] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21 reported cstate change: term changed from 0 to 1, leader changed from <none> to 442694a685c043e99fc1ad2e38b94b21 (127.5.61.2). New cstate: current_term: 1 leader_uuid: "442694a685c043e99fc1ad2e38b94b21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } health_report { overall_health: UNKNOWN } } }
I20260809 20:18:08.697745 14756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.691681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41994 (local address 127.5.61.2:44787)
0809 20:18:08.691855 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:08.691859 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:08.691875 (+ 16us) server_negotiation.cc:407] Connection header received
0809 20:18:08.691948 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:08.691951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:08.692031 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:08.692158 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:18:08.692472 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.693194 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.694911 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.695813 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.695947 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.696424 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:18:08.697186 (+ 762us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:18:08.697326 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:08.697440 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 20:18:08.697528 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 20:18:08.705477 14688 raft_consensus.cc:1261] T 1a42bfe6fd2d4a118730585bb0e46b23 P 54ee8649ea6247caa5df66298ed56767 [term 1 FOLLOWER]: Refusing update from remote peer 442694a685c043e99fc1ad2e38b94b21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:18:08.705559 14373 raft_consensus.cc:1261] T 1a42bfe6fd2d4a118730585bb0e46b23 P 34cffa13487e4d1f87816716f16f3cb3 [term 1 FOLLOWER]: Refusing update from remote peer 442694a685c043e99fc1ad2e38b94b21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:18:08.706463 14775 consensus_queue.cc:1035] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:18:08.706892 14769 consensus_queue.cc:1035] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:18:08.744638 14193 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:18:08.744942 14193 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:18:08.748710 14193 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d: Sending DeleteTablet for 3 replicas of tablet 1a42bfe6fd2d4a118730585bb0e46b23
I20260809 20:18:08.749823 14498 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a42bfe6fd2d4a118730585bb0e46b23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:18:08 UTC) from {username='oryx', principal='oryx/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:41986
I20260809 20:18:08.749922 14656 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a42bfe6fd2d4a118730585bb0e46b23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:18:08 UTC) from {username='oryx', principal='oryx/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:50730
I20260809 20:18:08.750799 14789 tablet_replica.cc:330] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21: stopping tablet replica
I20260809 20:18:08.750885 14345 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a42bfe6fd2d4a118730585bb0e46b23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:18:08 UTC) from {username='oryx', principal='oryx/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:50810
I20260809 20:18:08.751271 14789 raft_consensus.cc:2229] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21 [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:18:08.752022 14789 raft_consensus.cc:2258] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:18:08.752111 14193 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51270:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:18:08.753134 14193 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:18:08.754285 14789 ts_tablet_manager.cc:1905] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:18:08.759271 14791 tablet_replica.cc:330] T 1a42bfe6fd2d4a118730585bb0e46b23 P 34cffa13487e4d1f87816716f16f3cb3: stopping tablet replica
I20260809 20:18:08.759763 14791 raft_consensus.cc:2229] T 1a42bfe6fd2d4a118730585bb0e46b23 P 34cffa13487e4d1f87816716f16f3cb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:18:08.762946 14789 ts_tablet_manager.cc:1918] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:18:08.763260 14789 log.cc:1196] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/1a42bfe6fd2d4a118730585bb0e46b23
I20260809 20:18:08.763932 14498 tablet_service.cc:1467] Processing CreateTablet for tablet 258d6349536a45689b15a2c89fc81417 (DEFAULT_TABLE table=test-table [id=1ed40788cbb74672a412deb97bfada63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:18:08.764127 14789 ts_tablet_manager.cc:1939] T 1a42bfe6fd2d4a118730585bb0e46b23 P 442694a685c043e99fc1ad2e38b94b21: Deleting consensus metadata
I20260809 20:18:08.764297 14345 tablet_service.cc:1467] Processing CreateTablet for tablet 258d6349536a45689b15a2c89fc81417 (DEFAULT_TABLE table=test-table [id=1ed40788cbb74672a412deb97bfada63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:18:08.764480 14498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 258d6349536a45689b15a2c89fc81417. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:08.764780 14345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 258d6349536a45689b15a2c89fc81417. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:08.767745 14791 raft_consensus.cc:2258] T 1a42bfe6fd2d4a118730585bb0e46b23 P 34cffa13487e4d1f87816716f16f3cb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:18:08.768164 14655 tablet_service.cc:1467] Processing CreateTablet for tablet 258d6349536a45689b15a2c89fc81417 (DEFAULT_TABLE table=test-table [id=1ed40788cbb74672a412deb97bfada63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:18:08.768823 14655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 258d6349536a45689b15a2c89fc81417. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:08.770495 14791 ts_tablet_manager.cc:1905] T 1a42bfe6fd2d4a118730585bb0e46b23 P 34cffa13487e4d1f87816716f16f3cb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:18:08.771615 14762 tablet_bootstrap.cc:492] T 258d6349536a45689b15a2c89fc81417 P 34cffa13487e4d1f87816716f16f3cb3: Bootstrap starting.
I20260809 20:18:08.772163 14177 catalog_manager.cc:4841] TS 442694a685c043e99fc1ad2e38b94b21 (127.5.61.2:44787): tablet 1a42bfe6fd2d4a118730585bb0e46b23 (table test-table [id=4828d6c72d0d4a6fae75a8c97f9b1579]) successfully deleted
I20260809 20:18:08.773373 14762 tablet_bootstrap.cc:654] T 258d6349536a45689b15a2c89fc81417 P 34cffa13487e4d1f87816716f16f3cb3: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:08.777818 14791 ts_tablet_manager.cc:1918] T 1a42bfe6fd2d4a118730585bb0e46b23 P 34cffa13487e4d1f87816716f16f3cb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:18:08.778038 14791 log.cc:1196] T 1a42bfe6fd2d4a118730585bb0e46b23 P 34cffa13487e4d1f87816716f16f3cb3: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/1a42bfe6fd2d4a118730585bb0e46b23
I20260809 20:18:08.778656 14791 ts_tablet_manager.cc:1939] T 1a42bfe6fd2d4a118730585bb0e46b23 P 34cffa13487e4d1f87816716f16f3cb3: Deleting consensus metadata
I20260809 20:18:08.780439 14180 catalog_manager.cc:4841] TS 34cffa13487e4d1f87816716f16f3cb3 (127.5.61.1:40493): tablet 1a42bfe6fd2d4a118730585bb0e46b23 (table test-table [id=4828d6c72d0d4a6fae75a8c97f9b1579]) successfully deleted
I20260809 20:18:08.781454 14765 tablet_bootstrap.cc:492] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767: Bootstrap starting.
I20260809 20:18:08.783013 14765 tablet_bootstrap.cc:654] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:08.785286 14763 tablet_bootstrap.cc:492] T 258d6349536a45689b15a2c89fc81417 P 442694a685c043e99fc1ad2e38b94b21: Bootstrap starting.
I20260809 20:18:08.786716 14763 tablet_bootstrap.cc:654] T 258d6349536a45689b15a2c89fc81417 P 442694a685c043e99fc1ad2e38b94b21: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:08.789171 14763 tablet_bootstrap.cc:492] T 258d6349536a45689b15a2c89fc81417 P 442694a685c043e99fc1ad2e38b94b21: No bootstrap required, opened a new log
I20260809 20:18:08.789336 14763 ts_tablet_manager.cc:1397] T 258d6349536a45689b15a2c89fc81417 P 442694a685c043e99fc1ad2e38b94b21: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 20:18:08.789986 14763 raft_consensus.cc:348] T 258d6349536a45689b15a2c89fc81417 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.790175 14763 raft_consensus.cc:374] T 258d6349536a45689b15a2c89fc81417 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:08.790292 14763 raft_consensus.cc:729] T 258d6349536a45689b15a2c89fc81417 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442694a685c043e99fc1ad2e38b94b21, State: Initialized, Role: FOLLOWER
I20260809 20:18:08.790555 14763 consensus_queue.cc:260] T 258d6349536a45689b15a2c89fc81417 P 442694a685c043e99fc1ad2e38b94b21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.791070 14763 ts_tablet_manager.cc:1428] T 258d6349536a45689b15a2c89fc81417 P 442694a685c043e99fc1ad2e38b94b21: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 20:18:08.792821 14765 tablet_bootstrap.cc:492] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767: No bootstrap required, opened a new log
I20260809 20:18:08.792984 14765 ts_tablet_manager.cc:1397] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.004s
I20260809 20:18:08.793627 14765 raft_consensus.cc:348] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.793850 14765 raft_consensus.cc:374] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:08.793948 14765 raft_consensus.cc:729] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54ee8649ea6247caa5df66298ed56767, State: Initialized, Role: FOLLOWER
I20260809 20:18:08.794190 14765 consensus_queue.cc:260] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.795483 14765 ts_tablet_manager.cc:1428] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 20:18:08.796255 14790 tablet_replica.cc:330] T 1a42bfe6fd2d4a118730585bb0e46b23 P 54ee8649ea6247caa5df66298ed56767: stopping tablet replica
I20260809 20:18:08.796986 14790 raft_consensus.cc:2229] T 1a42bfe6fd2d4a118730585bb0e46b23 P 54ee8649ea6247caa5df66298ed56767 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:18:08.798347 14790 raft_consensus.cc:2258] T 1a42bfe6fd2d4a118730585bb0e46b23 P 54ee8649ea6247caa5df66298ed56767 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:18:08.798677 14762 tablet_bootstrap.cc:492] T 258d6349536a45689b15a2c89fc81417 P 34cffa13487e4d1f87816716f16f3cb3: No bootstrap required, opened a new log
I20260809 20:18:08.798869 14762 ts_tablet_manager.cc:1397] T 258d6349536a45689b15a2c89fc81417 P 34cffa13487e4d1f87816716f16f3cb3: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.026s
I20260809 20:18:08.799746 14762 raft_consensus.cc:348] T 258d6349536a45689b15a2c89fc81417 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.800024 14762 raft_consensus.cc:374] T 258d6349536a45689b15a2c89fc81417 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:08.800158 14762 raft_consensus.cc:729] T 258d6349536a45689b15a2c89fc81417 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34cffa13487e4d1f87816716f16f3cb3, State: Initialized, Role: FOLLOWER
I20260809 20:18:08.800500 14762 consensus_queue.cc:260] T 258d6349536a45689b15a2c89fc81417 P 34cffa13487e4d1f87816716f16f3cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.801020 14790 ts_tablet_manager.cc:1905] T 1a42bfe6fd2d4a118730585bb0e46b23 P 54ee8649ea6247caa5df66298ed56767: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:18:08.801608 14762 ts_tablet_manager.cc:1428] T 258d6349536a45689b15a2c89fc81417 P 34cffa13487e4d1f87816716f16f3cb3: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260809 20:18:08.807021 14790 ts_tablet_manager.cc:1918] T 1a42bfe6fd2d4a118730585bb0e46b23 P 54ee8649ea6247caa5df66298ed56767: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:18:08.807184 14790 log.cc:1196] T 1a42bfe6fd2d4a118730585bb0e46b23 P 54ee8649ea6247caa5df66298ed56767: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/1a42bfe6fd2d4a118730585bb0e46b23
I20260809 20:18:08.807727 14790 ts_tablet_manager.cc:1939] T 1a42bfe6fd2d4a118730585bb0e46b23 P 54ee8649ea6247caa5df66298ed56767: Deleting consensus metadata
I20260809 20:18:08.809154 14180 catalog_manager.cc:4841] TS 54ee8649ea6247caa5df66298ed56767 (127.5.61.3:41589): tablet 1a42bfe6fd2d4a118730585bb0e46b23 (table test-table [id=4828d6c72d0d4a6fae75a8c97f9b1579]) successfully deleted
I20260809 20:18:08.812493 14770 raft_consensus.cc:482] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:18:08.812682 14770 raft_consensus.cc:504] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.814385 14770 leader_election.cc:288] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34cffa13487e4d1f87816716f16f3cb3 (127.5.61.1:40493), 442694a685c043e99fc1ad2e38b94b21 (127.5.61.2:44787)
I20260809 20:18:08.821290 14754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.815408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:36731 (local address 127.5.61.1:40493)
0809 20:18:08.815622 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:08.815627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:08.815642 (+ 15us) server_negotiation.cc:407] Connection header received
0809 20:18:08.815700 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:08.815725 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:08.815805 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:08.815893 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:08.816589 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.817314 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.819248 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.820280 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.820414 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.820510 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:08.820869 (+ 359us) server_negotiation.cc:299] Negotiation successful
0809 20:18:08.821003 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 20:18:08.821337 14795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.814714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:40493 (local address 127.5.61.3:36731)
0809 20:18:08.815345 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:08.815361 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:08.815474 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:08.816054 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:08.816061 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:08.816089 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:08.816416 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:08.816427 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.817451 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.817458 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:08.819077 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:08.819089 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.820504 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.820511 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:08.820689 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.820724 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 20:18:08.820826 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 20:18:08.820964 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"mutex_wait_us":51,"thread_start_us":325,"threads_started":1}
I20260809 20:18:08.821985 14373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "258d6349536a45689b15a2c89fc81417" candidate_uuid: "54ee8649ea6247caa5df66298ed56767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34cffa13487e4d1f87816716f16f3cb3" is_pre_election: true
I20260809 20:18:08.822227 14373 raft_consensus.cc:2454] T 258d6349536a45689b15a2c89fc81417 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54ee8649ea6247caa5df66298ed56767 in term 0.
I20260809 20:18:08.822764 14605 leader_election.cc:302] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34cffa13487e4d1f87816716f16f3cb3, 54ee8649ea6247caa5df66298ed56767; no voters:
I20260809 20:18:08.823177 14770 raft_consensus.cc:2781] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:18:08.823302 14770 raft_consensus.cc:482] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:18:08.823395 14770 raft_consensus.cc:3037] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:08.823487 14756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.815202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:44363 (local address 127.5.61.2:44787)
0809 20:18:08.815423 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:08.815442 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:08.815461 (+ 19us) server_negotiation.cc:407] Connection header received
0809 20:18:08.815526 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:08.815530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:08.815644 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:08.815749 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:08.816163 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.817353 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.819925 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.821102 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.821238 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.821338 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:08.823004 (+ 1666us) server_negotiation.cc:299] Negotiation successful
0809 20:18:08.823136 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260809 20:18:08.825531 14770 raft_consensus.cc:504] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.826296 14611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.814635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:44787 (local address 127.5.61.3:44363)
0809 20:18:08.814839 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:08.814855 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:08.814967 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:08.815778 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:08.815783 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:08.815818 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:08.816012 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:08.816024 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.817538 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.817545 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:08.819733 (+ 2188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:08.819748 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.822582 (+ 2834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.822589 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:08.822800 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.822853 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 20:18:08.822977 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 20:18:08.825966 (+ 2989us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":76}
I20260809 20:18:08.826844 14373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "258d6349536a45689b15a2c89fc81417" candidate_uuid: "54ee8649ea6247caa5df66298ed56767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34cffa13487e4d1f87816716f16f3cb3"
I20260809 20:18:08.827059 14373 raft_consensus.cc:3037] T 258d6349536a45689b15a2c89fc81417 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:08.827361 14526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "258d6349536a45689b15a2c89fc81417" candidate_uuid: "54ee8649ea6247caa5df66298ed56767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442694a685c043e99fc1ad2e38b94b21"
I20260809 20:18:08.827417 14528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "258d6349536a45689b15a2c89fc81417" candidate_uuid: "54ee8649ea6247caa5df66298ed56767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442694a685c043e99fc1ad2e38b94b21" is_pre_election: true
I20260809 20:18:08.827874 14526 raft_consensus.cc:3037] T 258d6349536a45689b15a2c89fc81417 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:08.826355 14770 leader_election.cc:288] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767 [CANDIDATE]: Term 1 election: Requested vote from peers 34cffa13487e4d1f87816716f16f3cb3 (127.5.61.1:40493), 442694a685c043e99fc1ad2e38b94b21 (127.5.61.2:44787)
I20260809 20:18:08.830193 14373 raft_consensus.cc:2454] T 258d6349536a45689b15a2c89fc81417 P 34cffa13487e4d1f87816716f16f3cb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54ee8649ea6247caa5df66298ed56767 in term 1.
I20260809 20:18:08.830694 14605 leader_election.cc:302] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34cffa13487e4d1f87816716f16f3cb3, 54ee8649ea6247caa5df66298ed56767; no voters:
I20260809 20:18:08.831290 14770 raft_consensus.cc:2781] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:18:08.831580 14526 raft_consensus.cc:2454] T 258d6349536a45689b15a2c89fc81417 P 442694a685c043e99fc1ad2e38b94b21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54ee8649ea6247caa5df66298ed56767 in term 1.
I20260809 20:18:08.831745 14770 raft_consensus.cc:686] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767 [term 1 LEADER]: Becoming Leader. State: Replica: 54ee8649ea6247caa5df66298ed56767, State: Running, Role: LEADER
I20260809 20:18:08.832440 14770 consensus_queue.cc:237] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.836025 14195 catalog_manager.cc:5495] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767 reported cstate change: term changed from 0 to 1, leader changed from <none> to 54ee8649ea6247caa5df66298ed56767 (127.5.61.3). New cstate: current_term: 1 leader_uuid: "54ee8649ea6247caa5df66298ed56767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } health_report { overall_health: HEALTHY } } }
Aug 09 20:18:08 dist-test-slave-6k22 krb5kdc[14145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306686, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.62@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM
I20260809 20:18:08.885573 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.870304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51282 (local address 127.5.61.62:37709)
0809 20:18:08.870550 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:08.870557 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:08.871106 (+ 549us) server_negotiation.cc:407] Connection header received
0809 20:18:08.871250 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:08.871255 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:08.871330 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:08.871452 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:08.872673 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.873821 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.875028 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.875723 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.875864 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.879270 (+ 3406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:08.879277 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:08.879282 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:08.879320 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:08.882581 (+ 3261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:08.883346 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:08.883351 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:08.883354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:08.883597 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:08.883991 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:08.883997 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:08.884001 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:08.884252 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:08.884460 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:08.884918 (+ 458us) server_negotiation.cc:299] Negotiation successful
0809 20:18:08.885200 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":96,"server-negotiator.queue_time_us":111}
I20260809 20:18:08.885594 14803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.870304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:37709 (local address 127.0.0.1:51282)
0809 20:18:08.870991 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:08.871010 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:08.871137 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:08.871485 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:08.871488 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:08.872250 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:08.872513 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:08.872521 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.873951 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.873954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:08.874879 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:08.874889 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.875897 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.875902 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:08.876008 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.876683 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:08.876699 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:08.879077 (+ 2378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:08.882707 (+ 3630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:08.882717 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:08.882722 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:08.883204 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:08.883712 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:08.883716 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:08.883718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:08.883856 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:08.884448 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:08.884455 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:08.884609 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 20:18:08.884793 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 20:18:08.885080 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":394,"threads_started":1}
I20260809 20:18:08.890456 14195 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:51282:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 20:18:08.891462 14195 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:18:08.898463 14345 tablet_service.cc:1467] Processing CreateTablet for tablet 5a3029a8a052497d8ff79f4c3d4846b1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ad478cefa9ae487d805b53e3e0520805]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 20:18:08.898670 14498 tablet_service.cc:1467] Processing CreateTablet for tablet 5a3029a8a052497d8ff79f4c3d4846b1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ad478cefa9ae487d805b53e3e0520805]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 20:18:08.899025 14345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a3029a8a052497d8ff79f4c3d4846b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:08.899242 14498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a3029a8a052497d8ff79f4c3d4846b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:08.905194 14762 tablet_bootstrap.cc:492] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3: Bootstrap starting.
I20260809 20:18:08.906719 14762 tablet_bootstrap.cc:654] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:08.907215 14763 tablet_bootstrap.cc:492] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21: Bootstrap starting.
I20260809 20:18:08.908206 14655 tablet_service.cc:1467] Processing CreateTablet for tablet 5a3029a8a052497d8ff79f4c3d4846b1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ad478cefa9ae487d805b53e3e0520805]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 20:18:08.908756 14655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a3029a8a052497d8ff79f4c3d4846b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:08.909025 14763 tablet_bootstrap.cc:654] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:08.924424 14763 tablet_bootstrap.cc:492] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21: No bootstrap required, opened a new log
I20260809 20:18:08.924682 14763 ts_tablet_manager.cc:1397] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21: Time spent bootstrapping tablet: real 0.018s user 0.015s sys 0.001s
I20260809 20:18:08.925877 14765 tablet_bootstrap.cc:492] T 5a3029a8a052497d8ff79f4c3d4846b1 P 54ee8649ea6247caa5df66298ed56767: Bootstrap starting.
I20260809 20:18:08.925733 14763 raft_consensus.cc:348] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.926116 14763 raft_consensus.cc:374] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:08.926260 14763 raft_consensus.cc:729] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442694a685c043e99fc1ad2e38b94b21, State: Initialized, Role: FOLLOWER
I20260809 20:18:08.926653 14763 consensus_queue.cc:260] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.927641 14765 tablet_bootstrap.cc:654] T 5a3029a8a052497d8ff79f4c3d4846b1 P 54ee8649ea6247caa5df66298ed56767: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:08.927194 14784 tablet_replica.cc:439] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } } }
I20260809 20:18:08.927744 14769 raft_consensus.cc:482] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:18:08.927898 14769 raft_consensus.cc:504] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.927779 14784 tablet_replica.cc:442] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 20:18:08.928429 14763 ts_tablet_manager.cc:1428] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260809 20:18:08.928606 14769 leader_election.cc:288] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34cffa13487e4d1f87816716f16f3cb3 (127.5.61.1:40493), 54ee8649ea6247caa5df66298ed56767 (127.5.61.3:41589)
I20260809 20:18:08.929339 14373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3029a8a052497d8ff79f4c3d4846b1" candidate_uuid: "442694a685c043e99fc1ad2e38b94b21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34cffa13487e4d1f87816716f16f3cb3" is_pre_election: true
I20260809 20:18:08.929481 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3029a8a052497d8ff79f4c3d4846b1" candidate_uuid: "442694a685c043e99fc1ad2e38b94b21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54ee8649ea6247caa5df66298ed56767" is_pre_election: true
W20260809 20:18:08.931010 14445 leader_election.cc:341] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 54ee8649ea6247caa5df66298ed56767 (127.5.61.3:41589): Illegal state: must be running to vote when last-logged opid is not known
W20260809 20:18:08.931334 14445 leader_election.cc:341] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34cffa13487e4d1f87816716f16f3cb3 (127.5.61.1:40493): Illegal state: must be running to vote when last-logged opid is not known
I20260809 20:18:08.931473 14445 leader_election.cc:302] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 442694a685c043e99fc1ad2e38b94b21; no voters: 34cffa13487e4d1f87816716f16f3cb3, 54ee8649ea6247caa5df66298ed56767
I20260809 20:18:08.931851 14769 raft_consensus.cc:2726] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 20:18:08.941509 14765 tablet_bootstrap.cc:492] T 5a3029a8a052497d8ff79f4c3d4846b1 P 54ee8649ea6247caa5df66298ed56767: No bootstrap required, opened a new log
I20260809 20:18:08.941684 14765 ts_tablet_manager.cc:1397] T 5a3029a8a052497d8ff79f4c3d4846b1 P 54ee8649ea6247caa5df66298ed56767: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.005s
I20260809 20:18:08.941854 14762 tablet_bootstrap.cc:492] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3: No bootstrap required, opened a new log
I20260809 20:18:08.942066 14762 ts_tablet_manager.cc:1397] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3: Time spent bootstrapping tablet: real 0.037s user 0.005s sys 0.007s
I20260809 20:18:08.942502 14765 raft_consensus.cc:348] T 5a3029a8a052497d8ff79f4c3d4846b1 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.942770 14765 raft_consensus.cc:374] T 5a3029a8a052497d8ff79f4c3d4846b1 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:08.942870 14765 raft_consensus.cc:729] T 5a3029a8a052497d8ff79f4c3d4846b1 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54ee8649ea6247caa5df66298ed56767, State: Initialized, Role: FOLLOWER
I20260809 20:18:08.942981 14762 raft_consensus.cc:348] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.943229 14762 raft_consensus.cc:374] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:08.943146 14765 consensus_queue.cc:260] T 5a3029a8a052497d8ff79f4c3d4846b1 P 54ee8649ea6247caa5df66298ed56767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.943331 14762 raft_consensus.cc:729] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34cffa13487e4d1f87816716f16f3cb3, State: Initialized, Role: FOLLOWER
I20260809 20:18:08.943609 14762 consensus_queue.cc:260] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.943583 14770 tablet_replica.cc:439] T 5a3029a8a052497d8ff79f4c3d4846b1 P 54ee8649ea6247caa5df66298ed56767: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } } }
I20260809 20:18:08.943914 14770 tablet_replica.cc:442] T 5a3029a8a052497d8ff79f4c3d4846b1 P 54ee8649ea6247caa5df66298ed56767: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 20:18:08.944204 14762 ts_tablet_manager.cc:1428] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 20:18:08.944350 14765 ts_tablet_manager.cc:1428] T 5a3029a8a052497d8ff79f4c3d4846b1 P 54ee8649ea6247caa5df66298ed56767: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 20:18:08.944315 14768 tablet_replica.cc:439] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } } }
I20260809 20:18:08.944718 14768 tablet_replica.cc:442] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 20:18:08.979319 14768 raft_consensus.cc:482] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:18:08.979579 14768 raft_consensus.cc:504] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.981209 14768 leader_election.cc:288] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442694a685c043e99fc1ad2e38b94b21 (127.5.61.2:44787), 54ee8649ea6247caa5df66298ed56767 (127.5.61.3:41589)
I20260809 20:18:08.989784 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.981634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:33925 (local address 127.5.61.3:41589)
0809 20:18:08.981888 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:08.981897 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:08.982986 (+ 1089us) server_negotiation.cc:407] Connection header received
0809 20:18:08.983293 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:08.983298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:08.983372 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:08.983451 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:08.984280 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.985475 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.988017 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.988801 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.988966 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.989080 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:08.989367 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 20:18:08.989501 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 20:18:08.989953 14810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.981648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:41589 (local address 127.5.61.1:33925)
0809 20:18:08.982888 (+ 1240us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:08.982905 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:08.983109 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:08.983539 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:08.983544 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:08.983618 (+ 74us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:08.984079 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:08.984089 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.985622 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.985628 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:08.987815 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:08.987826 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.989026 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.989032 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:08.989197 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.989223 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 20:18:08.989330 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 20:18:08.989459 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1120,"mutex_wait_us":3,"spinlock_wait_cycles":2816,"thread_start_us":955,"threads_started":1}
I20260809 20:18:08.990499 14756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.981884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:56975 (local address 127.5.61.2:44787)
0809 20:18:08.982147 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:08.982153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:08.982657 (+ 504us) server_negotiation.cc:407] Connection header received
0809 20:18:08.983030 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:08.983036 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:08.983197 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:08.983305 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:08.984218 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.985160 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.987949 (+ 2789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.989266 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.989400 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.989495 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:08.990084 (+ 589us) server_negotiation.cc:299] Negotiation successful
0809 20:18:08.990237 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 20:18:08.990669 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3029a8a052497d8ff79f4c3d4846b1" candidate_uuid: "34cffa13487e4d1f87816716f16f3cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54ee8649ea6247caa5df66298ed56767" is_pre_election: true
I20260809 20:18:08.990969 14688 raft_consensus.cc:2454] T 5a3029a8a052497d8ff79f4c3d4846b1 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34cffa13487e4d1f87816716f16f3cb3 in term 0.
I20260809 20:18:08.991220 14808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:08.981449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:44787 (local address 127.5.61.1:56975)
0809 20:18:08.982517 (+ 1068us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:08.982544 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:08.982851 (+ 307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:08.983344 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:08.983348 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:08.983386 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:08.983986 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:08.984000 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.985310 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.985315 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:08.987768 (+ 2453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:08.987784 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:08.989659 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:08.989666 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:08.989872 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:08.989923 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 20:18:08.990744 (+ 821us) client_negotiation.cc:240] Negotiation successful
0809 20:18:08.990871 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":890,"mutex_wait_us":62,"spinlock_wait_cycles":18176,"thread_start_us":764,"threads_started":1}
I20260809 20:18:08.991462 14291 leader_election.cc:302] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34cffa13487e4d1f87816716f16f3cb3, 54ee8649ea6247caa5df66298ed56767; no voters:
I20260809 20:18:08.991825 14526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3029a8a052497d8ff79f4c3d4846b1" candidate_uuid: "34cffa13487e4d1f87816716f16f3cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442694a685c043e99fc1ad2e38b94b21" is_pre_election: true
I20260809 20:18:08.991974 14768 raft_consensus.cc:2781] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:18:08.992110 14768 raft_consensus.cc:482] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:18:08.992138 14526 raft_consensus.cc:2454] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34cffa13487e4d1f87816716f16f3cb3 in term 0.
I20260809 20:18:08.992200 14768 raft_consensus.cc:3037] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:08.995395 14768 raft_consensus.cc:504] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:08.996090 14768 leader_election.cc:288] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3 [CANDIDATE]: Term 1 election: Requested vote from peers 442694a685c043e99fc1ad2e38b94b21 (127.5.61.2:44787), 54ee8649ea6247caa5df66298ed56767 (127.5.61.3:41589)
I20260809 20:18:08.996511 14526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3029a8a052497d8ff79f4c3d4846b1" candidate_uuid: "34cffa13487e4d1f87816716f16f3cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442694a685c043e99fc1ad2e38b94b21"
I20260809 20:18:08.996546 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3029a8a052497d8ff79f4c3d4846b1" candidate_uuid: "34cffa13487e4d1f87816716f16f3cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54ee8649ea6247caa5df66298ed56767"
I20260809 20:18:08.996729 14526 raft_consensus.cc:3037] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:08.996778 14688 raft_consensus.cc:3037] T 5a3029a8a052497d8ff79f4c3d4846b1 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:08.998960 14526 raft_consensus.cc:2454] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34cffa13487e4d1f87816716f16f3cb3 in term 1.
I20260809 20:18:08.999392 14288 leader_election.cc:302] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34cffa13487e4d1f87816716f16f3cb3, 442694a685c043e99fc1ad2e38b94b21; no voters:
I20260809 20:18:08.999712 14768 raft_consensus.cc:2781] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:18:08.999856 14688 raft_consensus.cc:2454] T 5a3029a8a052497d8ff79f4c3d4846b1 P 54ee8649ea6247caa5df66298ed56767 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34cffa13487e4d1f87816716f16f3cb3 in term 1.
I20260809 20:18:09.000312 14768 raft_consensus.cc:686] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3 [term 1 LEADER]: Becoming Leader. State: Replica: 34cffa13487e4d1f87816716f16f3cb3, State: Running, Role: LEADER
I20260809 20:18:09.000933 14768 consensus_queue.cc:237] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:09.002847 14811 tablet_replica.cc:439] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3: TxnStatusTablet state changed. Reason: New leader 34cffa13487e4d1f87816716f16f3cb3. Latest consensus state: current_term: 1 leader_uuid: "34cffa13487e4d1f87816716f16f3cb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } } }
I20260809 20:18:09.003301 14811 tablet_replica.cc:442] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3: This TxnStatusTablet replica's current role is: LEADER
I20260809 20:18:09.004361 14813 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 20:18:09.004590 14813 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 20:18:09.004781 14193 catalog_manager.cc:5495] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34cffa13487e4d1f87816716f16f3cb3 (127.5.61.1). New cstate: current_term: 1 leader_uuid: "34cffa13487e4d1f87816716f16f3cb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } health_report { overall_health: UNKNOWN } } }
I20260809 20:18:09.049058 14754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:09.041569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50822 (local address 127.5.61.1:40493)
0809 20:18:09.041743 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:09.041751 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:09.041890 (+ 139us) server_negotiation.cc:407] Connection header received
0809 20:18:09.042146 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:09.042151 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:09.042267 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:09.042437 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:18:09.042848 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.044058 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.046172 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.046847 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.047053 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:09.047655 (+ 602us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:18:09.048385 (+ 730us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:18:09.048541 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:09.048690 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 20:18:09.048776 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 20:18:09.049075 14803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:09.041535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:40493 (local address 127.0.0.1:50822)
0809 20:18:09.041754 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:09.041781 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:09.041949 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:09.042476 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:09.042482 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:09.042497 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 20:18:09.042701 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:09.042712 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.044205 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.044210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:09.045910 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:09.045926 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.047077 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.047085 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:09.047342 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:09.047416 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 20:18:09.048508 (+ 1092us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 20:18:09.048537 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 20:18:09.048630 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 20:18:09.048779 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 20:18:09.053768 14688 raft_consensus.cc:1261] T 5a3029a8a052497d8ff79f4c3d4846b1 P 54ee8649ea6247caa5df66298ed56767 [term 1 FOLLOWER]: Refusing update from remote peer 34cffa13487e4d1f87816716f16f3cb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:18:09.054011 14526 raft_consensus.cc:1261] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21 [term 1 FOLLOWER]: Refusing update from remote peer 34cffa13487e4d1f87816716f16f3cb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:18:09.054672 14768 consensus_queue.cc:1035] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:18:09.055474 14811 consensus_queue.cc:1035] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:18:09.060267 14770 tablet_replica.cc:439] T 5a3029a8a052497d8ff79f4c3d4846b1 P 54ee8649ea6247caa5df66298ed56767: TxnStatusTablet state changed. Reason: New leader 34cffa13487e4d1f87816716f16f3cb3. Latest consensus state: current_term: 1 leader_uuid: "34cffa13487e4d1f87816716f16f3cb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } } }
I20260809 20:18:09.060575 14770 tablet_replica.cc:442] T 5a3029a8a052497d8ff79f4c3d4846b1 P 54ee8649ea6247caa5df66298ed56767: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 20:18:09.061086 14811 tablet_replica.cc:439] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "34cffa13487e4d1f87816716f16f3cb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } } }
I20260809 20:18:09.061298 14811 tablet_replica.cc:442] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3: This TxnStatusTablet replica's current role is: LEADER
I20260809 20:18:09.062589 14780 mvcc.cc:205] Tried to move back new op lower bound from 7316712198356656128 to 7316712198151147520. Current Snapshot: MvccSnapshot[applied={T|T < 7316712198356656128 or (T in {7316712198356656128})}]
I20260809 20:18:09.063951 14779 mvcc.cc:205] Tried to move back new op lower bound from 7316712198356656128 to 7316712198151147520. Current Snapshot: MvccSnapshot[applied={T|T < 7316712198356656128 or (T in {7316712198356656128})}]
I20260809 20:18:09.064646 14769 tablet_replica.cc:439] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21: TxnStatusTablet state changed. Reason: New leader 34cffa13487e4d1f87816716f16f3cb3. Latest consensus state: current_term: 1 leader_uuid: "34cffa13487e4d1f87816716f16f3cb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } } }
I20260809 20:18:09.064953 14769 tablet_replica.cc:442] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 20:18:09.067600 14770 tablet_replica.cc:439] T 5a3029a8a052497d8ff79f4c3d4846b1 P 54ee8649ea6247caa5df66298ed56767: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "34cffa13487e4d1f87816716f16f3cb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } } }
I20260809 20:18:09.067884 14770 tablet_replica.cc:442] T 5a3029a8a052497d8ff79f4c3d4846b1 P 54ee8649ea6247caa5df66298ed56767: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 20:18:09.074807 14768 tablet_replica.cc:439] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "34cffa13487e4d1f87816716f16f3cb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } } }
I20260809 20:18:09.075091 14768 tablet_replica.cc:442] T 5a3029a8a052497d8ff79f4c3d4846b1 P 34cffa13487e4d1f87816716f16f3cb3: This TxnStatusTablet replica's current role is: LEADER
I20260809 20:18:09.076756 14778 mvcc.cc:205] Tried to move back new op lower bound from 7316712198356656128 to 7316712198151147520. Current Snapshot: MvccSnapshot[applied={T|T < 7316712198356656128}]
I20260809 20:18:09.077621 14769 tablet_replica.cc:439] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "34cffa13487e4d1f87816716f16f3cb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } } }
I20260809 20:18:09.077859 14769 tablet_replica.cc:442] T 5a3029a8a052497d8ff79f4c3d4846b1 P 442694a685c043e99fc1ad2e38b94b21: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 20:18:09.082980 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:09.071889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50742 (local address 127.5.61.3:41589)
0809 20:18:09.072047 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:09.072052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:09.072070 (+ 18us) server_negotiation.cc:407] Connection header received
0809 20:18:09.072152 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:09.072159 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:09.072285 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:09.072446 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:18:09.072871 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.073886 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.079684 (+ 5798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.080388 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.080506 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:09.081132 (+ 626us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:18:09.081832 (+ 700us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:18:09.081961 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:09.082615 (+ 654us) server_negotiation.cc:299] Negotiation successful
0809 20:18:09.082696 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 20:18:09.102595 14754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:09.094293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:39747 (local address 127.5.61.1:40493)
0809 20:18:09.094575 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:09.094581 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:09.096194 (+ 1613us) server_negotiation.cc:407] Connection header received
0809 20:18:09.096388 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:09.096394 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:09.096480 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:09.096569 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:18:09.097169 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.098292 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.100373 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.101112 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.101241 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:09.101769 (+ 528us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:18:09.101895 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:18:09.101983 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:09.102164 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 20:18:09.102243 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 20:18:09.102676 14820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:09.094113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:40493 (local address 127.5.61.3:39747)
0809 20:18:09.096070 (+ 1957us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:09.096090 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:09.096242 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:09.096612 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:09.096617 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:09.096629 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 20:18:09.096972 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:09.096985 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.098471 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.098477 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:09.100213 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:09.100228 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.101315 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.101321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:09.101513 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:09.101584 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 20:18:09.102020 (+ 436us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 20:18:09.102032 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 20:18:09.102123 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 20:18:09.102263 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1792,"thread_start_us":1644,"threads_started":1}
I20260809 20:18:09.115708 14526 raft_consensus.cc:1261] T 258d6349536a45689b15a2c89fc81417 P 442694a685c043e99fc1ad2e38b94b21 [term 1 FOLLOWER]: Refusing update from remote peer 54ee8649ea6247caa5df66298ed56767: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:18:09.116401 14796 consensus_queue.cc:1035] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:18:09.131207 14373 raft_consensus.cc:1261] T 258d6349536a45689b15a2c89fc81417 P 34cffa13487e4d1f87816716f16f3cb3 [term 1 FOLLOWER]: Refusing update from remote peer 54ee8649ea6247caa5df66298ed56767: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:18:09.131799 14796 consensus_queue.cc:1035] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:18:09.200110 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:09.181831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:51949 (local address 127.5.61.3:41589)
0809 20:18:09.182034 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:09.182039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:09.187983 (+ 5944us) server_negotiation.cc:407] Connection header received
0809 20:18:09.188204 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:09.188213 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:09.188324 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:09.188431 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:18:09.193214 (+ 4783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.194212 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.196908 (+ 2696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.197917 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.198058 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:09.198991 (+ 933us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:18:09.199178 (+ 187us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:18:09.199266 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:09.199508 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 20:18:09.199638 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 20:18:09.201040 14827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:09.181352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:41589 (local address 127.5.61.1:51949)
0809 20:18:09.185868 (+ 4516us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:09.187907 (+ 2039us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:09.188067 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:09.192484 (+ 4417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:09.192497 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:09.192518 (+ 21us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 20:18:09.192985 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:09.193001 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.194427 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.194432 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:09.196691 (+ 2259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:09.196712 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.198301 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.198310 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:09.198602 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:09.198710 (+ 108us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 20:18:09.199342 (+ 632us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 20:18:09.199357 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 20:18:09.200365 (+ 1008us) client_negotiation.cc:240] Negotiation successful
0809 20:18:09.200570 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3623,"thread_start_us":3484,"threads_started":1}
I20260809 20:18:09.249655 14193 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:18:09.249905 14193 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:18:09.253450 14193 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d: Sending DeleteTablet for 3 replicas of tablet 258d6349536a45689b15a2c89fc81417
I20260809 20:18:09.254477 14655 tablet_service.cc:1514] Processing DeleteTablet for tablet 258d6349536a45689b15a2c89fc81417 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:18:09 UTC) from {username='oryx', principal='oryx/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:50730
I20260809 20:18:09.254724 14790 tablet_replica.cc:330] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767: stopping tablet replica
I20260809 20:18:09.254913 14790 raft_consensus.cc:2229] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767 [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:18:09.255359 14790 raft_consensus.cc:2258] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:18:09.256915 14498 tablet_service.cc:1514] Processing DeleteTablet for tablet 258d6349536a45689b15a2c89fc81417 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:18:09 UTC) from {username='oryx', principal='oryx/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:41986
I20260809 20:18:09.257203 14789 tablet_replica.cc:330] T 258d6349536a45689b15a2c89fc81417 P 442694a685c043e99fc1ad2e38b94b21: stopping tablet replica
I20260809 20:18:09.257386 14789 raft_consensus.cc:2229] T 258d6349536a45689b15a2c89fc81417 P 442694a685c043e99fc1ad2e38b94b21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:18:09.257586 14789 raft_consensus.cc:2258] T 258d6349536a45689b15a2c89fc81417 P 442694a685c043e99fc1ad2e38b94b21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:18:09.258463 14790 ts_tablet_manager.cc:1905] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:18:09.261540 14789 ts_tablet_manager.cc:1905] T 258d6349536a45689b15a2c89fc81417 P 442694a685c043e99fc1ad2e38b94b21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:18:09.276046 14345 tablet_service.cc:1514] Processing DeleteTablet for tablet 258d6349536a45689b15a2c89fc81417 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:18:09 UTC) from {username='oryx', principal='oryx/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:50810
I20260809 20:18:09.276369 14791 tablet_replica.cc:330] T 258d6349536a45689b15a2c89fc81417 P 34cffa13487e4d1f87816716f16f3cb3: stopping tablet replica
I20260809 20:18:09.276603 14791 raft_consensus.cc:2229] T 258d6349536a45689b15a2c89fc81417 P 34cffa13487e4d1f87816716f16f3cb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:18:09.276870 14791 raft_consensus.cc:2258] T 258d6349536a45689b15a2c89fc81417 P 34cffa13487e4d1f87816716f16f3cb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:18:09.278952 14791 ts_tablet_manager.cc:1905] T 258d6349536a45689b15a2c89fc81417 P 34cffa13487e4d1f87816716f16f3cb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:18:09.289907 14791 ts_tablet_manager.cc:1918] T 258d6349536a45689b15a2c89fc81417 P 34cffa13487e4d1f87816716f16f3cb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 20:18:09.290093 14791 log.cc:1196] T 258d6349536a45689b15a2c89fc81417 P 34cffa13487e4d1f87816716f16f3cb3: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/258d6349536a45689b15a2c89fc81417
I20260809 20:18:09.290374 14789 ts_tablet_manager.cc:1918] T 258d6349536a45689b15a2c89fc81417 P 442694a685c043e99fc1ad2e38b94b21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 20:18:09.290578 14791 ts_tablet_manager.cc:1939] T 258d6349536a45689b15a2c89fc81417 P 34cffa13487e4d1f87816716f16f3cb3: Deleting consensus metadata
I20260809 20:18:09.290616 14789 log.cc:1196] T 258d6349536a45689b15a2c89fc81417 P 442694a685c043e99fc1ad2e38b94b21: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/258d6349536a45689b15a2c89fc81417
I20260809 20:18:09.291126 14789 ts_tablet_manager.cc:1939] T 258d6349536a45689b15a2c89fc81417 P 442694a685c043e99fc1ad2e38b94b21: Deleting consensus metadata
I20260809 20:18:09.291926 14180 catalog_manager.cc:4841] TS 34cffa13487e4d1f87816716f16f3cb3 (127.5.61.1:40493): tablet 258d6349536a45689b15a2c89fc81417 (table test-table [id=1ed40788cbb74672a412deb97bfada63]) successfully deleted
I20260809 20:18:09.292476 14177 catalog_manager.cc:4841] TS 442694a685c043e99fc1ad2e38b94b21 (127.5.61.2:44787): tablet 258d6349536a45689b15a2c89fc81417 (table test-table [id=1ed40788cbb74672a412deb97bfada63]) successfully deleted
I20260809 20:18:09.293073 14790 ts_tablet_manager.cc:1918] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 20:18:09.293256 14790 log.cc:1196] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/258d6349536a45689b15a2c89fc81417
I20260809 20:18:09.293695 14790 ts_tablet_manager.cc:1939] T 258d6349536a45689b15a2c89fc81417 P 54ee8649ea6247caa5df66298ed56767: Deleting consensus metadata
I20260809 20:18:09.294718 14180 catalog_manager.cc:4841] TS 54ee8649ea6247caa5df66298ed56767 (127.5.61.3:41589): tablet 258d6349536a45689b15a2c89fc81417 (table test-table [id=1ed40788cbb74672a412deb97bfada63]) successfully deleted
Aug 09 20:18:09 dist-test-slave-6k22 krb5kdc[14145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 20:18:09 dist-test-slave-6k22 krb5kdc[14145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:09 dist-test-slave-6k22 krb5kdc[14145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 20:18:09.428223 14840 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:09.386922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51290 (local address 127.5.61.62:37709)
0809 20:18:09.396010 (+ 9088us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:09.396017 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:09.408295 (+ 12278us) server_negotiation.cc:407] Connection header received
0809 20:18:09.408546 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:09.408555 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:09.408656 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:09.408790 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:09.410878 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.412214 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.415720 (+ 3506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.416464 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.416592 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:09.427832 (+ 11240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51290: BlockingRecv error: recv got EOF from 127.0.0.1:51290 (error 108)
Metrics: {"server-negotiator.queue_time_us":8940,"thread_start_us":8791,"threads_started":1}
Aug 09 20:18:09 dist-test-slave-6k22 krb5kdc[14145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM
I20260809 20:18:09.493333 14840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:09.472330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51300 (local address 127.5.61.62:37709)
0809 20:18:09.472579 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:09.472587 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:09.473178 (+ 591us) server_negotiation.cc:407] Connection header received
0809 20:18:09.473386 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:09.473393 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:09.473485 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:09.473613 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:09.475456 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.476744 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.478742 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.479455 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.479638 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:09.485076 (+ 5438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:09.485084 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:09.485091 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:09.485146 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:09.489245 (+ 4099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:09.490170 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:09.490176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:09.490179 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:09.490370 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:09.490976 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:09.490981 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:09.490985 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:09.491397 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:09.491718 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:09.492713 (+ 995us) server_negotiation.cc:299] Negotiation successful
0809 20:18:09.492935 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 20:18:09.497807 14193 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:18:09.498616 14193 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:18:09.506145 14345 tablet_service.cc:1467] Processing CreateTablet for tablet 74a3e76e8ade4bbcb1740397ecdf0aa1 (DEFAULT_TABLE table=test-table [id=ff9332d3d3f641be9227e00da4806f56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:18:09.506639 14345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74a3e76e8ade4bbcb1740397ecdf0aa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:09.506724 14498 tablet_service.cc:1467] Processing CreateTablet for tablet 74a3e76e8ade4bbcb1740397ecdf0aa1 (DEFAULT_TABLE table=test-table [id=ff9332d3d3f641be9227e00da4806f56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:18:09.507215 14498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74a3e76e8ade4bbcb1740397ecdf0aa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:09.507963 14655 tablet_service.cc:1467] Processing CreateTablet for tablet 74a3e76e8ade4bbcb1740397ecdf0aa1 (DEFAULT_TABLE table=test-table [id=ff9332d3d3f641be9227e00da4806f56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:18:09.508378 14655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74a3e76e8ade4bbcb1740397ecdf0aa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:09.520963 14848 tablet_bootstrap.cc:492] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767: Bootstrap starting.
I20260809 20:18:09.522830 14849 tablet_bootstrap.cc:492] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 442694a685c043e99fc1ad2e38b94b21: Bootstrap starting.
I20260809 20:18:09.528268 14849 tablet_bootstrap.cc:654] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 442694a685c043e99fc1ad2e38b94b21: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:09.529734 14848 tablet_bootstrap.cc:654] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:09.533874 14847 tablet_bootstrap.cc:492] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 34cffa13487e4d1f87816716f16f3cb3: Bootstrap starting.
I20260809 20:18:09.536092 14847 tablet_bootstrap.cc:654] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 34cffa13487e4d1f87816716f16f3cb3: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:09.541018 14849 tablet_bootstrap.cc:492] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 442694a685c043e99fc1ad2e38b94b21: No bootstrap required, opened a new log
I20260809 20:18:09.541189 14849 ts_tablet_manager.cc:1397] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 442694a685c043e99fc1ad2e38b94b21: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.000s
I20260809 20:18:09.542013 14849 raft_consensus.cc:348] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:09.542294 14849 raft_consensus.cc:374] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:09.542387 14849 raft_consensus.cc:729] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442694a685c043e99fc1ad2e38b94b21, State: Initialized, Role: FOLLOWER
I20260809 20:18:09.542670 14849 consensus_queue.cc:260] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 442694a685c043e99fc1ad2e38b94b21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:09.544267 14849 ts_tablet_manager.cc:1428] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 442694a685c043e99fc1ad2e38b94b21: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 20:18:09.545701 14848 tablet_bootstrap.cc:492] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767: No bootstrap required, opened a new log
I20260809 20:18:09.545885 14848 ts_tablet_manager.cc:1397] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767: Time spent bootstrapping tablet: real 0.025s user 0.007s sys 0.000s
I20260809 20:18:09.546867 14848 raft_consensus.cc:348] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:09.547297 14848 raft_consensus.cc:374] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:09.547439 14848 raft_consensus.cc:729] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54ee8649ea6247caa5df66298ed56767, State: Initialized, Role: FOLLOWER
I20260809 20:18:09.547799 14848 consensus_queue.cc:260] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:09.549530 14847 tablet_bootstrap.cc:492] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 34cffa13487e4d1f87816716f16f3cb3: No bootstrap required, opened a new log
I20260809 20:18:09.549693 14847 ts_tablet_manager.cc:1397] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 34cffa13487e4d1f87816716f16f3cb3: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.000s
I20260809 20:18:09.550534 14847 raft_consensus.cc:348] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:09.550766 14848 ts_tablet_manager.cc:1428] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260809 20:18:09.550781 14847 raft_consensus.cc:374] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:09.550880 14847 raft_consensus.cc:729] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34cffa13487e4d1f87816716f16f3cb3, State: Initialized, Role: FOLLOWER
I20260809 20:18:09.551213 14847 consensus_queue.cc:260] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 34cffa13487e4d1f87816716f16f3cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:09.552420 14847 ts_tablet_manager.cc:1428] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 34cffa13487e4d1f87816716f16f3cb3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 20:18:09.612289 14828 raft_consensus.cc:482] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:18:09.612520 14828 raft_consensus.cc:504] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:09.613243 14828 leader_election.cc:288] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34cffa13487e4d1f87816716f16f3cb3 (127.5.61.1:40493), 442694a685c043e99fc1ad2e38b94b21 (127.5.61.2:44787)
I20260809 20:18:09.613828 14373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74a3e76e8ade4bbcb1740397ecdf0aa1" candidate_uuid: "54ee8649ea6247caa5df66298ed56767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34cffa13487e4d1f87816716f16f3cb3" is_pre_election: true
I20260809 20:18:09.614140 14373 raft_consensus.cc:2454] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54ee8649ea6247caa5df66298ed56767 in term 0.
I20260809 20:18:09.614285 14528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74a3e76e8ade4bbcb1740397ecdf0aa1" candidate_uuid: "54ee8649ea6247caa5df66298ed56767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442694a685c043e99fc1ad2e38b94b21" is_pre_election: true
I20260809 20:18:09.614590 14528 raft_consensus.cc:2454] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54ee8649ea6247caa5df66298ed56767 in term 0.
I20260809 20:18:09.614650 14605 leader_election.cc:302] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34cffa13487e4d1f87816716f16f3cb3, 54ee8649ea6247caa5df66298ed56767; no voters:
I20260809 20:18:09.614945 14828 raft_consensus.cc:2781] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:18:09.615079 14828 raft_consensus.cc:482] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:18:09.615190 14828 raft_consensus.cc:3037] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:09.618314 14828 raft_consensus.cc:504] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:09.618999 14828 leader_election.cc:288] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767 [CANDIDATE]: Term 1 election: Requested vote from peers 34cffa13487e4d1f87816716f16f3cb3 (127.5.61.1:40493), 442694a685c043e99fc1ad2e38b94b21 (127.5.61.2:44787)
I20260809 20:18:09.619863 14373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74a3e76e8ade4bbcb1740397ecdf0aa1" candidate_uuid: "54ee8649ea6247caa5df66298ed56767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34cffa13487e4d1f87816716f16f3cb3"
I20260809 20:18:09.620013 14528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74a3e76e8ade4bbcb1740397ecdf0aa1" candidate_uuid: "54ee8649ea6247caa5df66298ed56767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442694a685c043e99fc1ad2e38b94b21"
I20260809 20:18:09.620110 14373 raft_consensus.cc:3037] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:09.620262 14528 raft_consensus.cc:3037] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:09.623129 14373 raft_consensus.cc:2454] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 34cffa13487e4d1f87816716f16f3cb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54ee8649ea6247caa5df66298ed56767 in term 1.
I20260809 20:18:09.623203 14528 raft_consensus.cc:2454] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 442694a685c043e99fc1ad2e38b94b21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54ee8649ea6247caa5df66298ed56767 in term 1.
I20260809 20:18:09.623672 14601 leader_election.cc:302] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 442694a685c043e99fc1ad2e38b94b21, 54ee8649ea6247caa5df66298ed56767; no voters:
I20260809 20:18:09.624008 14828 raft_consensus.cc:2781] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:18:09.624223 14828 raft_consensus.cc:686] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767 [term 1 LEADER]: Becoming Leader. State: Replica: 54ee8649ea6247caa5df66298ed56767, State: Running, Role: LEADER
I20260809 20:18:09.624567 14828 consensus_queue.cc:237] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:09.627343 14191 catalog_manager.cc:5495] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767 reported cstate change: term changed from 0 to 1, leader changed from <none> to 54ee8649ea6247caa5df66298ed56767 (127.5.61.3). New cstate: current_term: 1 leader_uuid: "54ee8649ea6247caa5df66298ed56767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } health_report { overall_health: HEALTHY } } }
I20260809 20:18:09.649710 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:09.644395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50748 (local address 127.5.61.3:41589)
0809 20:18:09.644584 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:09.644590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:09.644774 (+ 184us) server_negotiation.cc:407] Connection header received
0809 20:18:09.644950 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:09.644953 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:09.645015 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:09.645086 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:18:09.645523 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.646371 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.648010 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.648450 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.648590 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:09.649033 (+ 443us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:18:09.649142 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:18:09.649246 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:09.649380 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 20:18:09.649456 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 20:18:09.652594 14528 raft_consensus.cc:1261] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 442694a685c043e99fc1ad2e38b94b21 [term 1 FOLLOWER]: Refusing update from remote peer 54ee8649ea6247caa5df66298ed56767: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:18:09.652863 14373 raft_consensus.cc:1261] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 34cffa13487e4d1f87816716f16f3cb3 [term 1 FOLLOWER]: Refusing update from remote peer 54ee8649ea6247caa5df66298ed56767: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:18:09.653357 14770 consensus_queue.cc:1035] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:18:09.653740 14828 consensus_queue.cc:1035] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:18:09.665906 14193 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:18:09.666203 14193 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:18:09.669818 14193 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d: Sending DeleteTablet for 3 replicas of tablet 74a3e76e8ade4bbcb1740397ecdf0aa1
I20260809 20:18:09.670842 14655 tablet_service.cc:1514] Processing DeleteTablet for tablet 74a3e76e8ade4bbcb1740397ecdf0aa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:18:09 UTC) from {username='oryx', principal='oryx/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:50730
I20260809 20:18:09.671221 14790 tablet_replica.cc:330] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767: stopping tablet replica
I20260809 20:18:09.671236 14498 tablet_service.cc:1514] Processing DeleteTablet for tablet 74a3e76e8ade4bbcb1740397ecdf0aa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:18:09 UTC) from {username='oryx', principal='oryx/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:41986
I20260809 20:18:09.671306 14497 tablet_service.cc:1514] Processing DeleteTablet for tablet 74a3e76e8ade4bbcb1740397ecdf0aa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:18:09 UTC) from {username='oryx', principal='oryx/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:41986
I20260809 20:18:09.671416 14790 raft_consensus.cc:2229] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767 [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:18:09.671533 14789 tablet_replica.cc:330] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 442694a685c043e99fc1ad2e38b94b21: stopping tablet replica
I20260809 20:18:09.671777 14790 raft_consensus.cc:2258] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:18:09.671903 14789 raft_consensus.cc:2229] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 442694a685c043e99fc1ad2e38b94b21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:18:09.672363 14789 raft_consensus.cc:2258] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 442694a685c043e99fc1ad2e38b94b21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:18:09.673239 14345 tablet_service.cc:1514] Processing DeleteTablet for tablet 74a3e76e8ade4bbcb1740397ecdf0aa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:18:09 UTC) from {username='oryx', principal='oryx/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:50810
I20260809 20:18:09.673534 14791 tablet_replica.cc:330] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 34cffa13487e4d1f87816716f16f3cb3: stopping tablet replica
I20260809 20:18:09.673331 14195 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 20:18:09.673686 14791 raft_consensus.cc:2229] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 34cffa13487e4d1f87816716f16f3cb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:18:09.673894 14791 raft_consensus.cc:2258] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 34cffa13487e4d1f87816716f16f3cb3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 20:18:09.673967 14195 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:18:09.674088 14790 ts_tablet_manager.cc:1905] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:18:09.675910 14791 ts_tablet_manager.cc:1905] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 34cffa13487e4d1f87816716f16f3cb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:18:09.680886 14345 tablet_service.cc:1467] Processing CreateTablet for tablet 129b2f17f8cd4a4fa296644a3cc5dfe1 (DEFAULT_TABLE table=test-table [id=047677c429ce42908cd5891e6f7376b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:18:09.681196 14655 tablet_service.cc:1467] Processing CreateTablet for tablet 129b2f17f8cd4a4fa296644a3cc5dfe1 (DEFAULT_TABLE table=test-table [id=047677c429ce42908cd5891e6f7376b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:18:09.681361 14345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 129b2f17f8cd4a4fa296644a3cc5dfe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:09.681612 14655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 129b2f17f8cd4a4fa296644a3cc5dfe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:09.682623 14791 ts_tablet_manager.cc:1918] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 34cffa13487e4d1f87816716f16f3cb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:18:09.682751 14791 log.cc:1196] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 34cffa13487e4d1f87816716f16f3cb3: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/74a3e76e8ade4bbcb1740397ecdf0aa1
I20260809 20:18:09.683421 14791 ts_tablet_manager.cc:1939] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 34cffa13487e4d1f87816716f16f3cb3: Deleting consensus metadata
I20260809 20:18:09.684686 14180 catalog_manager.cc:4841] TS 34cffa13487e4d1f87816716f16f3cb3 (127.5.61.1:40493): tablet 74a3e76e8ade4bbcb1740397ecdf0aa1 (table test-table [id=ff9332d3d3f641be9227e00da4806f56]) successfully deleted
I20260809 20:18:09.687492 14848 tablet_bootstrap.cc:492] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 54ee8649ea6247caa5df66298ed56767: Bootstrap starting.
I20260809 20:18:09.688153 14847 tablet_bootstrap.cc:492] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3: Bootstrap starting.
I20260809 20:18:09.688454 14497 tablet_service.cc:1467] Processing CreateTablet for tablet 129b2f17f8cd4a4fa296644a3cc5dfe1 (DEFAULT_TABLE table=test-table [id=047677c429ce42908cd5891e6f7376b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:18:09.688933 14497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 129b2f17f8cd4a4fa296644a3cc5dfe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:09.689428 14790 ts_tablet_manager.cc:1918] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:18:09.689554 14790 log.cc:1196] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/74a3e76e8ade4bbcb1740397ecdf0aa1
I20260809 20:18:09.689688 14847 tablet_bootstrap.cc:654] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:09.690001 14790 ts_tablet_manager.cc:1939] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 54ee8649ea6247caa5df66298ed56767: Deleting consensus metadata
I20260809 20:18:09.691740 14789 ts_tablet_manager.cc:1905] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 442694a685c043e99fc1ad2e38b94b21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:18:09.691860 14180 catalog_manager.cc:4841] TS 54ee8649ea6247caa5df66298ed56767 (127.5.61.3:41589): tablet 74a3e76e8ade4bbcb1740397ecdf0aa1 (table test-table [id=ff9332d3d3f641be9227e00da4806f56]) successfully deleted
I20260809 20:18:09.695824 14848 tablet_bootstrap.cc:654] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 54ee8649ea6247caa5df66298ed56767: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:09.699415 14789 ts_tablet_manager.cc:1918] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 442694a685c043e99fc1ad2e38b94b21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:18:09.699615 14789 log.cc:1196] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 442694a685c043e99fc1ad2e38b94b21: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/74a3e76e8ade4bbcb1740397ecdf0aa1
I20260809 20:18:09.700066 14789 ts_tablet_manager.cc:1939] T 74a3e76e8ade4bbcb1740397ecdf0aa1 P 442694a685c043e99fc1ad2e38b94b21: Deleting consensus metadata
I20260809 20:18:09.701493 14177 catalog_manager.cc:4841] TS 442694a685c043e99fc1ad2e38b94b21 (127.5.61.2:44787): tablet 74a3e76e8ade4bbcb1740397ecdf0aa1 (table test-table [id=ff9332d3d3f641be9227e00da4806f56]) successfully deleted
W20260809 20:18:09.702170 14177 catalog_manager.cc:4808] TS 442694a685c043e99fc1ad2e38b94b21 (127.5.61.2:44787): delete failed for tablet 74a3e76e8ade4bbcb1740397ecdf0aa1 because the tablet was not found. No further retry: Not found: Tablet not found: 74a3e76e8ade4bbcb1740397ecdf0aa1
I20260809 20:18:09.703161 14847 tablet_bootstrap.cc:492] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3: No bootstrap required, opened a new log
I20260809 20:18:09.703398 14847 ts_tablet_manager.cc:1397] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.006s
I20260809 20:18:09.704097 14849 tablet_bootstrap.cc:492] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 442694a685c043e99fc1ad2e38b94b21: Bootstrap starting.
I20260809 20:18:09.704360 14847 raft_consensus.cc:348] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:09.704707 14847 raft_consensus.cc:374] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:09.704855 14847 raft_consensus.cc:729] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34cffa13487e4d1f87816716f16f3cb3, State: Initialized, Role: FOLLOWER
I20260809 20:18:09.705204 14847 consensus_queue.cc:260] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:09.705735 14849 tablet_bootstrap.cc:654] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 442694a685c043e99fc1ad2e38b94b21: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:09.705801 14847 ts_tablet_manager.cc:1428] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 20:18:09.707149 14848 tablet_bootstrap.cc:492] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 54ee8649ea6247caa5df66298ed56767: No bootstrap required, opened a new log
I20260809 20:18:09.707355 14848 ts_tablet_manager.cc:1397] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 54ee8649ea6247caa5df66298ed56767: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.004s
I20260809 20:18:09.708177 14848 raft_consensus.cc:348] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:09.708472 14848 raft_consensus.cc:374] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:09.708581 14848 raft_consensus.cc:729] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54ee8649ea6247caa5df66298ed56767, State: Initialized, Role: FOLLOWER
I20260809 20:18:09.708899 14848 consensus_queue.cc:260] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 54ee8649ea6247caa5df66298ed56767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:09.709473 14848 ts_tablet_manager.cc:1428] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 54ee8649ea6247caa5df66298ed56767: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260809 20:18:09.713747 14849 tablet_bootstrap.cc:492] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 442694a685c043e99fc1ad2e38b94b21: No bootstrap required, opened a new log
I20260809 20:18:09.713941 14849 ts_tablet_manager.cc:1397] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 442694a685c043e99fc1ad2e38b94b21: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.004s
I20260809 20:18:09.714808 14849 raft_consensus.cc:348] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:09.715341 14849 raft_consensus.cc:374] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:09.715430 14849 raft_consensus.cc:729] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442694a685c043e99fc1ad2e38b94b21, State: Initialized, Role: FOLLOWER
I20260809 20:18:09.715682 14849 consensus_queue.cc:260] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 442694a685c043e99fc1ad2e38b94b21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:09.716217 14849 ts_tablet_manager.cc:1428] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 442694a685c043e99fc1ad2e38b94b21: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 20:18:09.723378 14811 raft_consensus.cc:482] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:18:09.723564 14811 raft_consensus.cc:504] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:09.724241 14811 leader_election.cc:288] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442694a685c043e99fc1ad2e38b94b21 (127.5.61.2:44787), 54ee8649ea6247caa5df66298ed56767 (127.5.61.3:41589)
I20260809 20:18:09.724638 14528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "129b2f17f8cd4a4fa296644a3cc5dfe1" candidate_uuid: "34cffa13487e4d1f87816716f16f3cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442694a685c043e99fc1ad2e38b94b21" is_pre_election: true
I20260809 20:18:09.724864 14528 raft_consensus.cc:2454] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34cffa13487e4d1f87816716f16f3cb3 in term 0.
I20260809 20:18:09.724852 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "129b2f17f8cd4a4fa296644a3cc5dfe1" candidate_uuid: "34cffa13487e4d1f87816716f16f3cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54ee8649ea6247caa5df66298ed56767" is_pre_election: true
I20260809 20:18:09.725138 14687 raft_consensus.cc:2454] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34cffa13487e4d1f87816716f16f3cb3 in term 0.
I20260809 20:18:09.725517 14288 leader_election.cc:302] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34cffa13487e4d1f87816716f16f3cb3, 442694a685c043e99fc1ad2e38b94b21; no voters:
I20260809 20:18:09.725847 14811 raft_consensus.cc:2781] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:18:09.725950 14811 raft_consensus.cc:482] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:18:09.726082 14811 raft_consensus.cc:3037] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:09.728317 14811 raft_consensus.cc:504] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:09.729671 14528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "129b2f17f8cd4a4fa296644a3cc5dfe1" candidate_uuid: "34cffa13487e4d1f87816716f16f3cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442694a685c043e99fc1ad2e38b94b21"
I20260809 20:18:09.729728 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "129b2f17f8cd4a4fa296644a3cc5dfe1" candidate_uuid: "34cffa13487e4d1f87816716f16f3cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54ee8649ea6247caa5df66298ed56767"
I20260809 20:18:09.729895 14528 raft_consensus.cc:3037] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 442694a685c043e99fc1ad2e38b94b21 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:09.729913 14687 raft_consensus.cc:3037] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 54ee8649ea6247caa5df66298ed56767 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:09.732923 14528 raft_consensus.cc:2454] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 442694a685c043e99fc1ad2e38b94b21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34cffa13487e4d1f87816716f16f3cb3 in term 1.
I20260809 20:18:09.733145 14687 raft_consensus.cc:2454] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 54ee8649ea6247caa5df66298ed56767 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34cffa13487e4d1f87816716f16f3cb3 in term 1.
I20260809 20:18:09.733577 14291 leader_election.cc:302] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34cffa13487e4d1f87816716f16f3cb3, 54ee8649ea6247caa5df66298ed56767; no voters:
I20260809 20:18:09.734086 14811 leader_election.cc:288] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3 [CANDIDATE]: Term 1 election: Requested vote from peers 442694a685c043e99fc1ad2e38b94b21 (127.5.61.2:44787), 54ee8649ea6247caa5df66298ed56767 (127.5.61.3:41589)
I20260809 20:18:09.734131 14861 raft_consensus.cc:2781] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:18:09.734354 14861 raft_consensus.cc:686] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3 [term 1 LEADER]: Becoming Leader. State: Replica: 34cffa13487e4d1f87816716f16f3cb3, State: Running, Role: LEADER
I20260809 20:18:09.734673 14861 consensus_queue.cc:237] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } }
I20260809 20:18:09.741348 14190 catalog_manager.cc:5495] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34cffa13487e4d1f87816716f16f3cb3 (127.5.61.1). New cstate: current_term: 1 leader_uuid: "34cffa13487e4d1f87816716f16f3cb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34cffa13487e4d1f87816716f16f3cb3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 40493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 } health_report { overall_health: UNKNOWN } } }
I20260809 20:18:09.764065 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:09.758181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50828 (local address 127.5.61.1:40493)
0809 20:18:09.758751 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:09.758760 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:09.758779 (+ 19us) server_negotiation.cc:407] Connection header received
0809 20:18:09.758842 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:09.758849 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:09.758951 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:09.759060 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:18:09.759477 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.760654 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.762070 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.762722 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.762881 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:09.763263 (+ 382us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:18:09.763389 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:18:09.763481 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:09.763634 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 20:18:09.763711 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":238,"threads_started":1}
I20260809 20:18:09.777310 14867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:09.771843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:40493 (local address 127.5.61.1:49615)
0809 20:18:09.772371 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:09.772386 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:09.772481 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:09.772803 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:09.772806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:09.772814 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 20:18:09.773014 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:09.773020 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.774314 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.774317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:09.775300 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:09.775310 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.776277 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.776281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:09.776396 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:09.776428 (+ 32us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 20:18:09.776827 (+ 399us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 20:18:09.776836 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 20:18:09.776910 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 20:18:09.777001 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":285,"threads_started":1}
I20260809 20:18:09.777340 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:09.771923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:49615 (local address 127.5.61.1:40493)
0809 20:18:09.772086 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:09.772094 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:09.772470 (+ 376us) server_negotiation.cc:407] Connection header received
0809 20:18:09.772588 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:09.772593 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:09.772678 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:09.772772 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:18:09.773168 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.774189 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.775447 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.776105 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.776245 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:09.776592 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:18:09.776713 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:18:09.776814 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:09.776961 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 20:18:09.777038 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 20:18:09.786137 14528 raft_consensus.cc:1261] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 442694a685c043e99fc1ad2e38b94b21 [term 1 FOLLOWER]: Refusing update from remote peer 34cffa13487e4d1f87816716f16f3cb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:18:09.786773 14861 consensus_queue.cc:1035] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "442694a685c043e99fc1ad2e38b94b21" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 44787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:18:09.802074 14687 raft_consensus.cc:1261] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 54ee8649ea6247caa5df66298ed56767 [term 1 FOLLOWER]: Refusing update from remote peer 34cffa13487e4d1f87816716f16f3cb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:18:09.804263 14811 consensus_queue.cc:1035] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54ee8649ea6247caa5df66298ed56767" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 41589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:18:09.855872 14190 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:18:09.856169 14190 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:18:09.859668 14190 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cef8dad6d8e0499aa793c78c9ab3f50d: Sending DeleteTablet for 3 replicas of tablet 129b2f17f8cd4a4fa296644a3cc5dfe1
I20260809 20:18:09.860558 14655 tablet_service.cc:1514] Processing DeleteTablet for tablet 129b2f17f8cd4a4fa296644a3cc5dfe1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:18:09 UTC) from {username='oryx', principal='oryx/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:50730
I20260809 20:18:09.860708 14345 tablet_service.cc:1514] Processing DeleteTablet for tablet 129b2f17f8cd4a4fa296644a3cc5dfe1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:18:09 UTC) from {username='oryx', principal='oryx/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:50810
I20260809 20:18:09.860852 14790 tablet_replica.cc:330] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 54ee8649ea6247caa5df66298ed56767: stopping tablet replica
I20260809 20:18:09.860998 14791 tablet_replica.cc:330] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3: stopping tablet replica
I20260809 20:18:09.861066 14790 raft_consensus.cc:2229] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 54ee8649ea6247caa5df66298ed56767 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:18:09.861215 14791 raft_consensus.cc:2229] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:18:09.861245 14790 raft_consensus.cc:2258] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 54ee8649ea6247caa5df66298ed56767 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:18:09.861693 14791 raft_consensus.cc:2258] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:18:09.862620 14497 tablet_service.cc:1514] Processing DeleteTablet for tablet 129b2f17f8cd4a4fa296644a3cc5dfe1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:18:09 UTC) from {username='oryx', principal='oryx/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:41986
I20260809 20:18:09.862843 14791 ts_tablet_manager.cc:1905] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:18:09.863721 14790 ts_tablet_manager.cc:1905] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 54ee8649ea6247caa5df66298ed56767: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:18:09.865885 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 14271
I20260809 20:18:09.866041 14789 tablet_replica.cc:330] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 442694a685c043e99fc1ad2e38b94b21: stopping tablet replica
I20260809 20:18:09.866266 14789 raft_consensus.cc:2229] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 442694a685c043e99fc1ad2e38b94b21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:18:09.866451 14789 raft_consensus.cc:2258] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 442694a685c043e99fc1ad2e38b94b21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:18:09.868312 14789 ts_tablet_manager.cc:1905] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 442694a685c043e99fc1ad2e38b94b21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:18:09.873212 14791 ts_tablet_manager.cc:1918] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 20:18:09.874555 14790 ts_tablet_manager.cc:1918] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 54ee8649ea6247caa5df66298ed56767: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 20:18:09.874703 14790 log.cc:1196] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 54ee8649ea6247caa5df66298ed56767: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/129b2f17f8cd4a4fa296644a3cc5dfe1
I20260809 20:18:09.874863 14791 log.cc:1196] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/129b2f17f8cd4a4fa296644a3cc5dfe1
I20260809 20:18:09.875133 14790 ts_tablet_manager.cc:1939] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 54ee8649ea6247caa5df66298ed56767: Deleting consensus metadata
I20260809 20:18:09.875257 14789 ts_tablet_manager.cc:1918] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 442694a685c043e99fc1ad2e38b94b21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 20:18:09.875416 14789 log.cc:1196] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 442694a685c043e99fc1ad2e38b94b21: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/129b2f17f8cd4a4fa296644a3cc5dfe1
I20260809 20:18:09.875456 14791 ts_tablet_manager.cc:1939] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 34cffa13487e4d1f87816716f16f3cb3: Deleting consensus metadata
I20260809 20:18:09.875901 14789 ts_tablet_manager.cc:1939] T 129b2f17f8cd4a4fa296644a3cc5dfe1 P 442694a685c043e99fc1ad2e38b94b21: Deleting consensus metadata
I20260809 20:18:09.876431 14180 catalog_manager.cc:4841] TS 54ee8649ea6247caa5df66298ed56767 (127.5.61.3:41589): tablet 129b2f17f8cd4a4fa296644a3cc5dfe1 (table test-table [id=047677c429ce42908cd5891e6f7376b4]) successfully deleted
I20260809 20:18:09.876955 14177 catalog_manager.cc:4841] TS 442694a685c043e99fc1ad2e38b94b21 (127.5.61.2:44787): tablet 129b2f17f8cd4a4fa296644a3cc5dfe1 (table test-table [id=047677c429ce42908cd5891e6f7376b4]) successfully deleted
I20260809 20:18:09.877339 14180 catalog_manager.cc:4841] TS 34cffa13487e4d1f87816716f16f3cb3 (127.5.61.1:40493): tablet 129b2f17f8cd4a4fa296644a3cc5dfe1 (table test-table [id=047677c429ce42908cd5891e6f7376b4]) successfully deleted
Aug 09 20:18:09 dist-test-slave-6k22 krb5kdc[14145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.61.1@KRBTEST.COM
I20260809 20:18:09.888693 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:09.867989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50832 (local address 127.5.61.1:40493)
0809 20:18:09.868203 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:09.868210 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:09.870765 (+ 2555us) server_negotiation.cc:407] Connection header received
0809 20:18:09.871036 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:09.871047 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:09.871136 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:09.871206 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:09.872977 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.874124 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.876050 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:09.876657 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:09.876769 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:09.881727 (+ 4958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:09.881744 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:09.881764 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:09.881819 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:09.885710 (+ 3891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:09.886391 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:09.886401 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:09.886404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:09.886610 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:09.886963 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:09.886969 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:09.886972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:09.887425 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:09.887796 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:09.888031 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 20:18:09.888241 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 20:18:09.889412 14415 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:10.023278 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 14271
I20260809 20:18:10.050513 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 14424
Aug 09 20:18:10 dist-test-slave-6k22 krb5kdc[14145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.61.2@KRBTEST.COM
I20260809 20:18:10.065982 14878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:10.051351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42004 (local address 127.5.61.2:44787)
0809 20:18:10.051915 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:10.051919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:10.051935 (+ 16us) server_negotiation.cc:407] Connection header received
0809 20:18:10.051992 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:10.051995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:10.052053 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:10.052144 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:10.053346 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:10.054095 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:10.055394 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:10.055855 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:10.055948 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:10.060160 (+ 4212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:10.060196 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:10.060217 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:10.060296 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:10.063505 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:10.064139 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:10.064145 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:10.064147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:10.064288 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:10.064592 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:10.064595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:10.064597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:10.064917 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:10.065135 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:10.065443 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 20:18:10.065598 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":316,"threads_started":1}
I20260809 20:18:10.066612 14571 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:10.194283 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 14424
I20260809 20:18:10.222458 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 14582
Aug 09 20:18:10 dist-test-slave-6k22 krb5kdc[14145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.61.3@KRBTEST.COM
I20260809 20:18:10.237018 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:10.223206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50758 (local address 127.5.61.3:41589)
0809 20:18:10.223686 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:10.223690 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:10.223705 (+ 15us) server_negotiation.cc:407] Connection header received
0809 20:18:10.223758 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:10.223762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:10.223822 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:10.223910 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:10.225104 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:10.225718 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:10.226826 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:10.227217 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:10.227343 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:10.231647 (+ 4304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:10.231666 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:10.231683 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:10.231718 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:10.234645 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:10.235204 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:10.235211 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:10.235213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:10.235342 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:10.235708 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:10.235711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:10.235713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:10.236054 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:10.236305 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:10.236492 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 20:18:10.236653 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":268,"threads_started":1}
I20260809 20:18:10.237686 14730 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:10.363734 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 14582
I20260809 20:18:10.389887 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 14158
I20260809 20:18:10.391218 14236 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:10.508076 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 14158
2026-08-09T20:18:10Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (5684 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:18:10 dist-test-slave-6k22 krb5kdc[14885](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:18:10 dist-test-slave-6k22 krb5kdc[14885](info): set up 2 sockets
Aug 09 20:18:10 dist-test-slave-6k22 krb5kdc[14885](info): commencing operation
krb5kdc: starting...
W20260809 20:18:11.688624 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:18:11 dist-test-slave-6k22 krb5kdc[14885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:18:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:18:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:18:11.925019 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:41029
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:34567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:41029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:18:12.248977 14899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:12.249396 14899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:12.249558 14899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:12.256467 14899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:18:12.256593 14899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:12.256680 14899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:12.256774 14899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:18:12.256843 14899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:18:12.266031 14899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:34567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:41029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:41029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:12.267119 14899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:12.268466 14899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:12.280014 14909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:12.280081 14912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:12.280105 14908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:12.281124 14899 server_base.cc:1034] running on GCE node
I20260809 20:18:12.281958 14899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:12.283295 14899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:12.284555 14899 hybrid_clock.cc:648] HybridClock initialized: now 1786306692284474 us; error 78 us; skew 500 ppm
I20260809 20:18:12.284919 14899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T20:18:12Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1850 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:18:12 dist-test-slave-6k22 krb5kdc[14920](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:18:12 dist-test-slave-6k22 krb5kdc[14920](info): set up 2 sockets
Aug 09 20:18:12 dist-test-slave-6k22 krb5kdc[14920](info): commencing operation
krb5kdc: starting...
W20260809 20:18:13.548336 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:18:13 dist-test-slave-6k22 krb5kdc[14920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:18:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:18:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:18:13.795267 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:38651
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:42167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:38651
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:18:14.117236 14934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:14.117542 14934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:14.117753 14934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:14.124895 14934 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:18:14.125000 14934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:14.125049 14934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:14.125192 14934 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:18:14.125262 14934 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:18:14.135664 14934 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:42167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:38651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:38651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:14.136965 14934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:14.138291 14934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:14.150182 14939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:14.150230 14940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:14.150142 14942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:14.151491 14934 server_base.cc:1034] running on GCE node
I20260809 20:18:14.152343 14934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:14.153746 14934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:14.155004 14934 hybrid_clock.cc:648] HybridClock initialized: now 1786306694154940 us; error 34 us; skew 500 ppm
I20260809 20:18:14.155407 14934 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:18:14 dist-test-slave-6k22 krb5kdc[14920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306694, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:18:14.158612 14934 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:18:14.160246 14934 webserver.cc:460] Webserver started at http://127.5.61.62:35929/ using document root <none> and password file <none>
I20260809 20:18:14.160928 14934 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:14.161096 14934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:14.161428 14934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:18:14.164466 14934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "4f3b3fc8c0ad4606860a8da41928cba1"
format_stamp: "Formatted at 2026-08-09 20:18:14 on dist-test-slave-6k22"
server_key: "89a0cb9a1a166cdc28bfe46b08565c3e"
server_key_iv: "8659640357da0a653d7584ca3bb78da7"
server_key_version: "encryptionkey@0"
I20260809 20:18:14.165266 14934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "4f3b3fc8c0ad4606860a8da41928cba1"
format_stamp: "Formatted at 2026-08-09 20:18:14 on dist-test-slave-6k22"
server_key: "89a0cb9a1a166cdc28bfe46b08565c3e"
server_key_iv: "8659640357da0a653d7584ca3bb78da7"
server_key_version: "encryptionkey@0"
I20260809 20:18:14.170403 14934 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 20:18:14.174718 14951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:14.176522 14934 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 20:18:14.176723 14934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "4f3b3fc8c0ad4606860a8da41928cba1"
format_stamp: "Formatted at 2026-08-09 20:18:14 on dist-test-slave-6k22"
server_key: "89a0cb9a1a166cdc28bfe46b08565c3e"
server_key_iv: "8659640357da0a653d7584ca3bb78da7"
server_key_version: "encryptionkey@0"
I20260809 20:18:14.176959 14934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:14.204309 14934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:14.208384 14934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:14.208706 14934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:14.228233 14934 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:38651
I20260809 20:18:14.228291 15006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:38651 every 8 connection(s)
I20260809 20:18:14.229655 14934 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:18:14.233276 15007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:14.238911 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 14934
I20260809 20:18:14.239138 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:18:14.240016 5364 external_mini_cluster.cc:1428] Setting key a38ae1b0303c46f60295ce41227c7614
I20260809 20:18:14.244779 15007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1: Bootstrap starting.
I20260809 20:18:14.247802 15007 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:14.248816 15007 log.cc:824] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1: Log is configured to *not* fsync() on all Append() calls
I20260809 20:18:14.251935 15007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1: No bootstrap required, opened a new log
Aug 09 20:18:14 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:18:14.262022 15007 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f3b3fc8c0ad4606860a8da41928cba1" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38651 } }
I20260809 20:18:14.262483 15007 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:14.262604 15007 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f3b3fc8c0ad4606860a8da41928cba1, State: Initialized, Role: FOLLOWER
I20260809 20:18:14.263247 15007 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f3b3fc8c0ad4606860a8da41928cba1" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38651 } }
I20260809 20:18:14.263525 15007 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:18:14.263685 15007 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:18:14.263887 15012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:14.242275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56312 (local address 127.5.61.62:38651)
0809 20:18:14.243115 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:14.243126 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:14.243163 (+ 37us) server_negotiation.cc:407] Connection header received
0809 20:18:14.244899 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:14.244931 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:14.245303 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:14.245669 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:14.247068 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:14.248473 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:14.250065 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:14.251037 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:14.251241 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:14.255303 (+ 4062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:14.255314 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:14.255331 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:14.255386 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:14.259441 (+ 4055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:14.260139 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:14.260152 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:14.260160 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:14.260408 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:14.260742 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:14.260747 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:14.260751 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:14.261420 (+ 669us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:14.261709 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:14.262023 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 20:18:14.262528 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":408,"threads_started":1}
I20260809 20:18:14.263907 15007 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:14.266702 15007 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f3b3fc8c0ad4606860a8da41928cba1" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38651 } }
I20260809 20:18:14.267141 15007 leader_election.cc:302] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f3b3fc8c0ad4606860a8da41928cba1; no voters:
I20260809 20:18:14.267732 15007 leader_election.cc:288] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:18:14.267874 15014 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:18:14.268585 15014 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [term 1 LEADER]: Becoming Leader. State: Replica: 4f3b3fc8c0ad4606860a8da41928cba1, State: Running, Role: LEADER
I20260809 20:18:14.269107 15014 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f3b3fc8c0ad4606860a8da41928cba1" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38651 } }
I20260809 20:18:14.269697 15007 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:18:14.272778 15016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f3b3fc8c0ad4606860a8da41928cba1. Latest consensus state: current_term: 1 leader_uuid: "4f3b3fc8c0ad4606860a8da41928cba1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f3b3fc8c0ad4606860a8da41928cba1" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38651 } } }
I20260809 20:18:14.272790 15015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f3b3fc8c0ad4606860a8da41928cba1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f3b3fc8c0ad4606860a8da41928cba1" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38651 } } }
I20260809 20:18:14.273269 15016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [sys.catalog]: This master's current role is: LEADER
I20260809 20:18:14.273286 15015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [sys.catalog]: This master's current role is: LEADER
I20260809 20:18:14.279804 15022 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:18:14.284538 15022 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:18:14.292716 15022 catalog_manager.cc:1347] Generated new cluster ID: 4e7e393730eb4892a06b23788e0411ec
I20260809 20:18:14.292838 15022 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:18:14.313192 15022 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:18:14.314360 15022 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:18:14.321933 15022 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1: Generated new TSK 0
I20260809 20:18:14.322690 15022 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.61.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
I20260809 20:18:14.384146 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:38651
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:42167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:18:14.711654 15036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:14.711905 15036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:14.711993 15036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:14.720412 15036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:14.720517 15036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:14.720623 15036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:18:14.730106 15036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:42167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:38651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:14.731287 15036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:14.732645 15036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:14.745641 15041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:14.745610 15045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:14.745687 15048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:14.746797 15036 server_base.cc:1034] running on GCE node
I20260809 20:18:14.747388 15036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:14.748311 15036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:14.749498 15036 hybrid_clock.cc:648] HybridClock initialized: now 1786306694749447 us; error 51 us; skew 500 ppm
I20260809 20:18:14.749790 15036 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:18:14 dist-test-slave-6k22 krb5kdc[14920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306694, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:18:14.753021 15036 init.cc:377] Logged in from keytab as kudu/127.5.61.1@KRBTEST.COM (short username kudu)
I20260809 20:18:14.754370 15036 webserver.cc:460] Webserver started at http://127.5.61.1:41235/ using document root <none> and password file <none>
I20260809 20:18:14.755051 15036 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:14.755199 15036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:14.755522 15036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:18:14.758476 15036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "32f8b94a2b8f40ed9164d1a9c30da789"
format_stamp: "Formatted at 2026-08-09 20:18:14 on dist-test-slave-6k22"
server_key: "5c3481d4a45598ba5cbe1f0b1898dfd3"
server_key_iv: "9ada555e754f5d8490d39e4a2b336c5f"
server_key_version: "encryptionkey@0"
I20260809 20:18:14.759253 15036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "32f8b94a2b8f40ed9164d1a9c30da789"
format_stamp: "Formatted at 2026-08-09 20:18:14 on dist-test-slave-6k22"
server_key: "5c3481d4a45598ba5cbe1f0b1898dfd3"
server_key_iv: "9ada555e754f5d8490d39e4a2b336c5f"
server_key_version: "encryptionkey@0"
I20260809 20:18:14.764631 15036 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260809 20:18:14.767941 15058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:14.769146 15036 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 20:18:14.769320 15036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "32f8b94a2b8f40ed9164d1a9c30da789"
format_stamp: "Formatted at 2026-08-09 20:18:14 on dist-test-slave-6k22"
server_key: "5c3481d4a45598ba5cbe1f0b1898dfd3"
server_key_iv: "9ada555e754f5d8490d39e4a2b336c5f"
server_key_version: "encryptionkey@0"
I20260809 20:18:14.769547 15036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:14.785614 15036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:14.789844 15036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:14.790131 15036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:14.791036 15036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:18:14.792349 15036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:18:14.792446 15036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:14.792588 15036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:18:14.792654 15036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:14.827545 15036 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:46095
I20260809 20:18:14.827632 15181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:46095 every 8 connection(s)
I20260809 20:18:14.828907 15036 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:18:14.838032 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 15036
I20260809 20:18:14.838259 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:18:14.838908 5364 external_mini_cluster.cc:1428] Setting key 761eabfe8e7fb2907694352132b2f5f9
Aug 09 20:18:14 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306694, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:18:14.848642 15184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:14.831591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:32855 (local address 127.5.61.62:38651)
0809 20:18:14.832057 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:14.832062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:14.832930 (+ 868us) server_negotiation.cc:407] Connection header received
0809 20:18:14.834760 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:14.834766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:14.834866 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:14.834985 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:14.836577 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:14.837326 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:14.839022 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:14.839435 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:14.839601 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:14.843019 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:14.843024 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:14.843028 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:14.843057 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:14.845508 (+ 2451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:14.846283 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:14.846288 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:14.846290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:14.846419 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:14.846867 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:14.846876 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:14.846878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:14.847122 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:14.847275 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:14.848048 (+ 773us) server_negotiation.cc:299] Negotiation successful
0809 20:18:14.848270 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":251,"threads_started":1}
I20260809 20:18:14.849574 15185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:14.832021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.1:32855)
0809 20:18:14.832756 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:14.832802 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:14.834497 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:14.835250 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:14.835258 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:14.835838 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:14.836403 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:14.836423 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:14.837521 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:14.837525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:14.838878 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:14.838890 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:14.839689 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:14.839697 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:14.839957 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:14.840657 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:14.840679 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:14.842802 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:14.845655 (+ 2853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:14.845664 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:14.845698 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:14.846143 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:14.846534 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:14.846539 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:14.846543 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:14.846733 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:14.847254 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:14.847262 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:14.847673 (+ 411us) client_negotiation.cc:769] Sending connection context
0809 20:18:14.847933 (+ 260us) client_negotiation.cc:240] Negotiation successful
0809 20:18:14.848452 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":306,"threads_started":1}
I20260809 20:18:14.851186 15182 heartbeater.cc:347] Connected to a master server at 127.5.61.62:38651
I20260809 20:18:14.851601 15182 heartbeater.cc:464] Registering TS with master...
I20260809 20:18:14.852339 15182 heartbeater.cc:511] Master 127.5.61.62:38651 requested a full tablet report, sending...
I20260809 20:18:14.854621 14969 ts_manager.cc:194] Registered new tserver with Master: 32f8b94a2b8f40ed9164d1a9c30da789 (127.5.61.1:46095)
I20260809 20:18:14.856823 14969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.1@KRBTEST.COM'} at 127.5.61.1:32855
WARNING: no policy specified for kudu/127.5.61.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
I20260809 20:18:14.898142 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:38651
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:42167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:18:15.233737 15188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:15.234112 15188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:15.234251 15188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:15.245693 15188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:15.245829 15188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:15.246013 15188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:18:15.256454 15188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:42167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:38651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:15.257614 15188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:15.258872 15188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:15.272169 15197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:15.272217 15195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:15.272421 15194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:15.273504 15188 server_base.cc:1034] running on GCE node
I20260809 20:18:15.274219 15188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:15.275199 15188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:15.276430 15188 hybrid_clock.cc:648] HybridClock initialized: now 1786306695276382 us; error 59 us; skew 500 ppm
I20260809 20:18:15.276758 15188 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:18:15 dist-test-slave-6k22 krb5kdc[14920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306695, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:18:15.280041 15188 init.cc:377] Logged in from keytab as kudu/127.5.61.2@KRBTEST.COM (short username kudu)
I20260809 20:18:15.281670 15188 webserver.cc:460] Webserver started at http://127.5.61.2:36371/ using document root <none> and password file <none>
I20260809 20:18:15.282478 15188 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:15.282630 15188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:15.283036 15188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:18:15.286145 15188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "30ca59060d664237be16be0a0727ba49"
format_stamp: "Formatted at 2026-08-09 20:18:15 on dist-test-slave-6k22"
server_key: "2e904740be4e1608f86f26debf4f2bc2"
server_key_iv: "eceb2e86f5fedef6d59eb3c8510b7368"
server_key_version: "encryptionkey@0"
I20260809 20:18:15.286960 15188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "30ca59060d664237be16be0a0727ba49"
format_stamp: "Formatted at 2026-08-09 20:18:15 on dist-test-slave-6k22"
server_key: "2e904740be4e1608f86f26debf4f2bc2"
server_key_iv: "eceb2e86f5fedef6d59eb3c8510b7368"
server_key_version: "encryptionkey@0"
I20260809 20:18:15.292251 15188 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 20:18:15.295674 15205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:15.296949 15188 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:18:15.297130 15188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "30ca59060d664237be16be0a0727ba49"
format_stamp: "Formatted at 2026-08-09 20:18:15 on dist-test-slave-6k22"
server_key: "2e904740be4e1608f86f26debf4f2bc2"
server_key_iv: "eceb2e86f5fedef6d59eb3c8510b7368"
server_key_version: "encryptionkey@0"
I20260809 20:18:15.297340 15188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:15.313616 15188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:15.318019 15188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:15.318323 15188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:15.319332 15188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:18:15.320719 15188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:18:15.320849 15188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:15.321010 15188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:18:15.321084 15188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:15.354969 15188 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:39029
I20260809 20:18:15.355014 15332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:39029 every 8 connection(s)
I20260809 20:18:15.356330 15188 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:18:15.363185 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 15188
I20260809 20:18:15.363368 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:18:15.363867 5364 external_mini_cluster.cc:1428] Setting key 04ba6d6a94643c22d2450cf4956501e8
Aug 09 20:18:15 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306695, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:18:15.376703 15337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:15.358849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:60731 (local address 127.5.61.62:38651)
0809 20:18:15.359273 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:15.359278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:15.360441 (+ 1163us) server_negotiation.cc:407] Connection header received
0809 20:18:15.362227 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:15.362232 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:15.362312 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:15.362410 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:15.364043 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:15.364759 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:15.366353 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:15.366791 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:15.366941 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:15.370486 (+ 3545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:15.370496 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:15.370499 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:15.370531 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:15.373446 (+ 2915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:15.374315 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:15.374327 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:15.374330 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:15.374459 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:15.374917 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:15.374921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:15.374923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:15.375183 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:15.375342 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:15.376102 (+ 760us) server_negotiation.cc:299] Negotiation successful
0809 20:18:15.376327 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":204,"threads_started":1}
I20260809 20:18:15.377745 15339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:15.359399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.2:60731)
0809 20:18:15.360259 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:15.360302 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:15.361984 (+ 1682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:15.362607 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:15.362615 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:15.363144 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:15.363863 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:15.363882 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:15.364939 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:15.364945 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:15.366183 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:15.366195 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:15.367029 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:15.367037 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:15.367262 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:15.367999 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:15.368021 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:15.370251 (+ 2230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:15.373601 (+ 3350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:15.373610 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:15.373628 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:15.374186 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:15.374578 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:15.374583 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:15.374587 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:15.374778 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:15.375319 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:15.375326 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:15.375711 (+ 385us) client_negotiation.cc:769] Sending connection context
0809 20:18:15.375990 (+ 279us) client_negotiation.cc:240] Negotiation successful
0809 20:18:15.376532 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":317,"threads_started":1}
I20260809 20:18:15.379220 15333 heartbeater.cc:347] Connected to a master server at 127.5.61.62:38651
I20260809 20:18:15.379549 15333 heartbeater.cc:464] Registering TS with master...
I20260809 20:18:15.380539 15333 heartbeater.cc:511] Master 127.5.61.62:38651 requested a full tablet report, sending...
I20260809 20:18:15.382218 14969 ts_manager.cc:194] Registered new tserver with Master: 30ca59060d664237be16be0a0727ba49 (127.5.61.2:39029)
I20260809 20:18:15.383522 14969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.2@KRBTEST.COM'} at 127.5.61.2:60731
WARNING: no policy specified for kudu/127.5.61.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
I20260809 20:18:15.422597 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:38651
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:42167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:18:15.757557 15342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:15.757843 15342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:15.757937 15342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:15.765049 15342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:15.765161 15342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:15.765267 15342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:18:15.774724 15342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:42167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:38651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:15.775849 15342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:15.777001 15342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:15.789829 15348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:15.789793 15349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:15.789855 15351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:15.790961 15342 server_base.cc:1034] running on GCE node
I20260809 20:18:15.791471 15342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:15.792444 15342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:15.793637 15342 hybrid_clock.cc:648] HybridClock initialized: now 1786306695793595 us; error 38 us; skew 500 ppm
I20260809 20:18:15.793895 15342 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:18:15 dist-test-slave-6k22 krb5kdc[14920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306695, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:18:15.797091 15342 init.cc:377] Logged in from keytab as kudu/127.5.61.3@KRBTEST.COM (short username kudu)
I20260809 20:18:15.798447 15342 webserver.cc:460] Webserver started at http://127.5.61.3:44001/ using document root <none> and password file <none>
I20260809 20:18:15.799124 15342 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:15.799260 15342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:15.799638 15342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:18:15.802665 15342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "8230bf6db4c9435cbf289fbead860d7a"
format_stamp: "Formatted at 2026-08-09 20:18:15 on dist-test-slave-6k22"
server_key: "199d9bfb628878da0eb69dbcec7cbf33"
server_key_iv: "b41869523b3afbe81b66369ab32b3bf3"
server_key_version: "encryptionkey@0"
I20260809 20:18:15.803428 15342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "8230bf6db4c9435cbf289fbead860d7a"
format_stamp: "Formatted at 2026-08-09 20:18:15 on dist-test-slave-6k22"
server_key: "199d9bfb628878da0eb69dbcec7cbf33"
server_key_iv: "b41869523b3afbe81b66369ab32b3bf3"
server_key_version: "encryptionkey@0"
I20260809 20:18:15.808461 15342 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 20:18:15.811820 15359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:15.813055 15342 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:18:15.813226 15342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "8230bf6db4c9435cbf289fbead860d7a"
format_stamp: "Formatted at 2026-08-09 20:18:15 on dist-test-slave-6k22"
server_key: "199d9bfb628878da0eb69dbcec7cbf33"
server_key_iv: "b41869523b3afbe81b66369ab32b3bf3"
server_key_version: "encryptionkey@0"
I20260809 20:18:15.813423 15342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:15.833134 15342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:15.837009 15342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:15.837304 15342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:15.838264 15342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:18:15.839663 15342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:18:15.839764 15342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:15.839933 15342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:18:15.840009 15342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:15.860033 15182 heartbeater.cc:503] Master 127.5.61.62:38651 was elected leader, sending a full tablet report...
I20260809 20:18:15.873832 15342 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:38439
I20260809 20:18:15.873888 15483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:38439 every 8 connection(s)
I20260809 20:18:15.875145 15342 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:18:15.875885 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 15342
I20260809 20:18:15.876082 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:18:15.876695 5364 external_mini_cluster.cc:1428] Setting key 33b7b1d148a252f0249cb796c6569519
Aug 09 20:18:15 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306695, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:18:15.895090 15487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:15.878038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:46449 (local address 127.5.61.62:38651)
0809 20:18:15.878562 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:15.878568 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:15.881813 (+ 3245us) server_negotiation.cc:407] Connection header received
0809 20:18:15.881894 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:15.881899 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:15.881971 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:15.882053 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:15.883583 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:15.884343 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:15.885812 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:15.886181 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:15.886277 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:15.889503 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:15.889508 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:15.889511 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:15.889539 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:15.892687 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:15.893302 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:15.893305 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:15.893307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:15.893418 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:15.893769 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:15.893772 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:15.893774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:15.894017 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:15.894116 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:15.894666 (+ 550us) server_negotiation.cc:299] Negotiation successful
0809 20:18:15.894844 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":245,"threads_started":1}
I20260809 20:18:15.896021 15488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:15.878769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.3:46449)
0809 20:18:15.879551 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:15.879659 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:15.881560 (+ 1901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:15.882310 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:15.882317 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:15.882878 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:15.883407 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:15.883425 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:15.884466 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:15.884470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:15.885691 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:15.885703 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:15.886358 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:15.886364 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:15.886547 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:15.887143 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:15.887169 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:15.889308 (+ 2139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:15.892804 (+ 3496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:15.892810 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:15.892820 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:15.893206 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:15.893506 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:15.893509 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:15.893511 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:15.893673 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:15.894111 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:15.894116 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:15.894374 (+ 258us) client_negotiation.cc:769] Sending connection context
0809 20:18:15.894585 (+ 211us) client_negotiation.cc:240] Negotiation successful
0809 20:18:15.894918 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":322,"threads_started":1}
I20260809 20:18:15.897259 15485 heartbeater.cc:347] Connected to a master server at 127.5.61.62:38651
I20260809 20:18:15.897540 15485 heartbeater.cc:464] Registering TS with master...
I20260809 20:18:15.898231 15485 heartbeater.cc:511] Master 127.5.61.62:38651 requested a full tablet report, sending...
I20260809 20:18:15.899551 14969 ts_manager.cc:194] Registered new tserver with Master: 8230bf6db4c9435cbf289fbead860d7a (127.5.61.3:38439)
I20260809 20:18:15.900425 14969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.3@KRBTEST.COM'} at 127.5.61.3:46449
I20260809 20:18:15.904320 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.61.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.61.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.61.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.1.keytab.
Entry for principal oryx/127.5.61.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.1.keytab.
WARNING: no policy specified for oryx/127.5.61.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.61.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.61.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.2.keytab.
Entry for principal oryx/127.5.61.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.2.keytab.
WARNING: no policy specified for oryx/127.5.61.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.61.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.61.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.3.keytab.
Entry for principal oryx/127.5.61.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.3.keytab.
I20260809 20:18:16.050284 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 15036
Aug 09 20:18:16 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.1@KRBTEST.COM
I20260809 20:18:16.067462 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:16.051525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40424 (local address 127.5.61.1:46095)
0809 20:18:16.052241 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:16.052248 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:16.052278 (+ 30us) server_negotiation.cc:407] Connection header received
0809 20:18:16.052451 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:16.052458 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:16.052722 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:16.052904 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:16.054822 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:16.056139 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:16.057318 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:16.057961 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:16.058079 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:16.062197 (+ 4118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:16.062208 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:16.062222 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:16.062254 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:16.064933 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:16.065531 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:16.065537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:16.065539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:16.065659 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:16.066012 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:16.066015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:16.066017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:16.066332 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:16.066554 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:16.066804 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 20:18:16.067023 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":394,"threads_started":1}
I20260809 20:18:16.068886 15176 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:16.150830 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 15036
I20260809 20:18:16.169904 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 15188
Aug 09 20:18:16 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.2@KRBTEST.COM
I20260809 20:18:16.189869 15499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:16.170861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51332 (local address 127.5.61.2:39029)
0809 20:18:16.171605 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:16.171620 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:16.171646 (+ 26us) server_negotiation.cc:407] Connection header received
0809 20:18:16.171714 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:16.171720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:16.171928 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:16.172058 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:16.173753 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:16.175630 (+ 1877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:16.177434 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:16.178409 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:16.178540 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:16.184149 (+ 5609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:16.184161 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:16.184175 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:16.184212 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:16.187149 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:16.187839 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:16.187846 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:16.187848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:16.187975 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:16.188325 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:16.188329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:16.188331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:16.188728 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:16.188985 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:16.189198 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 20:18:16.189391 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":399,"threads_started":1}
I20260809 20:18:16.191129 15328 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:16.273165 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 15188
I20260809 20:18:16.293679 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 15342
Aug 09 20:18:16 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.3@KRBTEST.COM
I20260809 20:18:16.309819 15501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:16.294887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33992 (local address 127.5.61.3:38439)
0809 20:18:16.295543 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:16.295550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:16.295632 (+ 82us) server_negotiation.cc:407] Connection header received
0809 20:18:16.295697 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:16.295703 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:16.295951 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:16.296116 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:16.297398 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:16.298450 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:16.299790 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:16.300377 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:16.300475 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:16.304793 (+ 4318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:16.304804 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:16.304817 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:16.304851 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:16.307383 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:16.308100 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:16.308107 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:16.308109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:16.308244 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:16.308562 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:16.308565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:16.308567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:16.308888 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:16.309106 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:16.309292 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 20:18:16.309468 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":343,"threads_started":1}
I20260809 20:18:16.311054 15479 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:16.393046 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 15342
I20260809 20:18:16.412292 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 14934
I20260809 20:18:16.413498 15002 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:16.516984 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 14934
I20260809 20:18:16.541554 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:38651
--webserver_interface=127.5.61.62
--webserver_port=35929
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:42167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:38651
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:18:16.863693 15504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:16.864029 15504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:16.864202 15504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:16.871134 15504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:18:16.871281 15504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:16.871399 15504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:16.871500 15504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:18:16.871582 15504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:18:16.881143 15504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:42167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:38651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:38651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=35929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:16.882443 15504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:16.883772 15504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:16.895324 15510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:16.895460 15509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:16.895853 15512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:16.896209 15504 server_base.cc:1034] running on GCE node
I20260809 20:18:16.896927 15504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:16.898156 15504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:16.899367 15504 hybrid_clock.cc:648] HybridClock initialized: now 1786306696899329 us; error 40 us; skew 500 ppm
I20260809 20:18:16.899763 15504 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:18:16 dist-test-slave-6k22 krb5kdc[14920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306696, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:18:16.902850 15504 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:18:16.904376 15504 webserver.cc:460] Webserver started at http://127.5.61.62:35929/ using document root <none> and password file <none>
I20260809 20:18:16.905030 15504 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:16.905187 15504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:16.910246 15504 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 20:18:16.912966 15520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:16.914161 15504 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 20:18:16.914369 15504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "4f3b3fc8c0ad4606860a8da41928cba1"
format_stamp: "Formatted at 2026-08-09 20:18:14 on dist-test-slave-6k22"
server_key: "89a0cb9a1a166cdc28bfe46b08565c3e"
server_key_iv: "8659640357da0a653d7584ca3bb78da7"
server_key_version: "encryptionkey@0"
I20260809 20:18:16.915252 15504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:16.937300 15504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:16.941509 15504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:16.941787 15504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:16.960407 15504 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:38651
I20260809 20:18:16.960441 15579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:38651 every 8 connection(s)
I20260809 20:18:16.961899 15504 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:18:16.966670 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 15504
I20260809 20:18:16.967536 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:46095
--local_ip_for_outbound_sockets=127.5.61.1
--tserver_master_addrs=127.5.61.62:38651
--webserver_port=41235
--webserver_interface=127.5.61.1
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:42167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.61.1
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.1.keytab with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 20:18:16.968775 15580 sys_catalog.cc:263] Verifying existing consensus state
I20260809 20:18:16.971990 15580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1: Bootstrap starting.
I20260809 20:18:16.996490 15580 log.cc:824] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1: Log is configured to *not* fsync() on all Append() calls
I20260809 20:18:17.007865 15580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 20:18:17.008512 15580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1: Bootstrap complete.
I20260809 20:18:17.021471 15580 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f3b3fc8c0ad4606860a8da41928cba1" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38651 } }
I20260809 20:18:17.022589 15580 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f3b3fc8c0ad4606860a8da41928cba1, State: Initialized, Role: FOLLOWER
I20260809 20:18:17.023267 15580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f3b3fc8c0ad4606860a8da41928cba1" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38651 } }
I20260809 20:18:17.023535 15580 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:18:17.023672 15580 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:18:17.023847 15580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [term 1 FOLLOWER]: Advancing to term 2
I20260809 20:18:17.029321 15580 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f3b3fc8c0ad4606860a8da41928cba1" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38651 } }
I20260809 20:18:17.029892 15580 leader_election.cc:302] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f3b3fc8c0ad4606860a8da41928cba1; no voters:
I20260809 20:18:17.030553 15580 leader_election.cc:288] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 20:18:17.030702 15588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 20:18:17.032064 15588 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [term 2 LEADER]: Becoming Leader. State: Replica: 4f3b3fc8c0ad4606860a8da41928cba1, State: Running, Role: LEADER
I20260809 20:18:17.032603 15588 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f3b3fc8c0ad4606860a8da41928cba1" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38651 } }
I20260809 20:18:17.032778 15580 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:18:17.035172 15591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f3b3fc8c0ad4606860a8da41928cba1. Latest consensus state: current_term: 2 leader_uuid: "4f3b3fc8c0ad4606860a8da41928cba1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f3b3fc8c0ad4606860a8da41928cba1" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38651 } } }
I20260809 20:18:17.035651 15591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [sys.catalog]: This master's current role is: LEADER
I20260809 20:18:17.035192 15590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4f3b3fc8c0ad4606860a8da41928cba1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f3b3fc8c0ad4606860a8da41928cba1" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38651 } } }
I20260809 20:18:17.036003 15590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1 [sys.catalog]: This master's current role is: LEADER
I20260809 20:18:17.047679 15596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:18:17.054256 15596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:18:17.056144 15596 catalog_manager.cc:1259] Loaded cluster ID: 4e7e393730eb4892a06b23788e0411ec
I20260809 20:18:17.056344 15596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:18:17.061574 15596 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:18:17.063464 15596 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4f3b3fc8c0ad4606860a8da41928cba1: Loaded TSK: 0
I20260809 20:18:17.065398 15596 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 20:18:17.346218 15585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:17.346527 15585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:17.346729 15585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:17.353907 15585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:17.354045 15585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:17.354200 15585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:18:17.363741 15585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:42167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.1.keytab
--principal=oryx/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:46095
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=41235
--tserver_master_addrs=127.5.61.62:38651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:17.365006 15585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:17.366297 15585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:17.381168 15612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:17.381863 15615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:17.382247 15613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:17.383191 15585 server_base.cc:1034] running on GCE node
I20260809 20:18:17.383965 15585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:17.385089 15585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:17.386318 15585 hybrid_clock.cc:648] HybridClock initialized: now 1786306697386265 us; error 41 us; skew 500 ppm
I20260809 20:18:17.386669 15585 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:18:17 dist-test-slave-6k22 krb5kdc[14920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306697, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:18:17.396270 15585 init.cc:377] Logged in from keytab as oryx/127.5.61.1@KRBTEST.COM (short username oryx)
I20260809 20:18:17.398054 15585 webserver.cc:460] Webserver started at http://127.5.61.1:41235/ using document root <none> and password file <none>
I20260809 20:18:17.398805 15585 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:17.398962 15585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:17.404817 15585 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 20:18:17.407899 15626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:17.409389 15585 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 20:18:17.409578 15585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "32f8b94a2b8f40ed9164d1a9c30da789"
format_stamp: "Formatted at 2026-08-09 20:18:14 on dist-test-slave-6k22"
server_key: "5c3481d4a45598ba5cbe1f0b1898dfd3"
server_key_iv: "9ada555e754f5d8490d39e4a2b336c5f"
server_key_version: "encryptionkey@0"
I20260809 20:18:17.410583 15585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:17.439705 15585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:17.444496 15585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:17.444942 15585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:17.445978 15585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:18:17.447431 15585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:18:17.447675 15585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:17.447839 15585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:18:17.447981 15585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:17.482682 15585 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:46095
I20260809 20:18:17.482774 15747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:46095 every 8 connection(s)
I20260809 20:18:17.484172 15585 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:18:17.489374 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 15585
I20260809 20:18:17.490049 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:39029
--local_ip_for_outbound_sockets=127.5.61.2
--tserver_master_addrs=127.5.61.62:38651
--webserver_port=36371
--webserver_interface=127.5.61.2
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:42167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.61.2
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.2.keytab with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 20:18:17 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:17 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 20:18:17.503394 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:17.487198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:37313 (local address 127.5.61.62:38651)
0809 20:18:17.488024 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:17.488039 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:17.488274 (+ 235us) server_negotiation.cc:407] Connection header received
0809 20:18:17.490610 (+ 2336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:17.490623 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:17.490932 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:17.491295 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:17.492809 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:17.494112 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:17.495705 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:17.496377 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:17.496597 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:17.502519 (+ 5922us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.1:37313: BlockingRecv error: recv got EOF from 127.5.61.1:37313 (error 108)
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":317,"threads_started":1}
I20260809 20:18:17.503489 15752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:17.487243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.1:37313)
0809 20:18:17.488102 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:17.488140 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:17.489918 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:17.491431 (+ 1513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:17.491442 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:17.492086 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:17.492607 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:17.492624 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:17.494303 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:17.494307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:17.495506 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:17.495518 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:17.496622 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:17.496629 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:17.496918 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:17.497625 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:17.497648 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:17.502213 (+ 4565us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":430,"threads_started":1}
W20260809 20:18:17.504698 15748 heartbeater.cc:644] Failed to heartbeat to 127.5.61.62:38651 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.61.62:38651: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Aug 09 20:18:17 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:17 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:17.513072 15752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:17.505354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.1:53837)
0809 20:18:17.505494 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:17.505519 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:17.505630 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:17.505975 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:17.505979 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:17.506308 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:17.506438 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:17.506445 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:17.507441 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:17.507446 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:17.508405 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:17.508416 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:17.509101 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:17.509107 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:17.509285 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:17.509877 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:17.509893 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:17.512789 (+ 2896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 20:18:17.513196 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:17.505517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:53837 (local address 127.5.61.62:38651)
0809 20:18:17.505724 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:17.505730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:17.505753 (+ 23us) server_negotiation.cc:407] Connection header received
0809 20:18:17.505793 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:17.505796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:17.505847 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:17.505949 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:17.506572 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:17.507295 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:17.508544 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:17.508948 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:17.509079 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:17.512978 (+ 3899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.1:53837: BlockingRecv error: recv got EOF from 127.5.61.1:53837 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 20:18:17 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:17 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:17.523264 15752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:17.513845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.1:59555)
0809 20:18:17.514062 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:17.514083 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:17.514192 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:17.514489 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:17.514497 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:17.514956 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:17.515199 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:17.515215 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:17.516424 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:17.516432 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:17.518241 (+ 1809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:17.518259 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:17.518969 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:17.518975 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:17.519150 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:17.519791 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:17.519813 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:17.522979 (+ 3166us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":74}
W20260809 20:18:17.523361 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:17.513991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:59555 (local address 127.5.61.62:38651)
0809 20:18:17.514185 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:17.514189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:17.514205 (+ 16us) server_negotiation.cc:407] Connection header received
0809 20:18:17.514302 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:17.514305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:17.514367 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:17.514442 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:17.515354 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:17.516271 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:17.518386 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:17.518825 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:17.518917 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:17.523157 (+ 4240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.1:59555: BlockingRecv error: recv got EOF from 127.5.61.1:59555 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 20:18:17.523710 15748 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 20:18:17.833180 15753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:17.833444 15753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:17.833630 15753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:17.840868 15753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:17.840970 15753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:17.841146 15753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:18:17.850987 15753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:42167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.2.keytab
--principal=oryx/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:39029
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=36371
--tserver_master_addrs=127.5.61.62:38651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:17.852245 15753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:17.853528 15753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:17.866318 15758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:17.866326 15761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:17.866369 15759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:17.867619 15753 server_base.cc:1034] running on GCE node
I20260809 20:18:17.868183 15753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:17.869065 15753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:17.870262 15753 hybrid_clock.cc:648] HybridClock initialized: now 1786306697870220 us; error 37 us; skew 500 ppm
I20260809 20:18:17.870566 15753 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:18:17 dist-test-slave-6k22 krb5kdc[14920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306697, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:18:17.873762 15753 init.cc:377] Logged in from keytab as oryx/127.5.61.2@KRBTEST.COM (short username oryx)
I20260809 20:18:17.875209 15753 webserver.cc:460] Webserver started at http://127.5.61.2:36371/ using document root <none> and password file <none>
I20260809 20:18:17.875979 15753 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:17.876140 15753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:17.881378 15753 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 20:18:17.884080 15769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:17.885344 15753 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 20:18:17.885528 15753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "30ca59060d664237be16be0a0727ba49"
format_stamp: "Formatted at 2026-08-09 20:18:15 on dist-test-slave-6k22"
server_key: "2e904740be4e1608f86f26debf4f2bc2"
server_key_iv: "eceb2e86f5fedef6d59eb3c8510b7368"
server_key_version: "encryptionkey@0"
I20260809 20:18:17.886482 15753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:17.903811 15753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:17.907850 15753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:17.908113 15753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:17.909024 15753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:18:17.910283 15753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:18:17.910408 15753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:17.910528 15753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:18:17.910619 15753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:17.943601 15753 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:39029
I20260809 20:18:17.943715 15891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:39029 every 8 connection(s)
I20260809 20:18:17.945021 15753 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:18:17.947731 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 15753
I20260809 20:18:17.948433 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:38439
--local_ip_for_outbound_sockets=127.5.61.3
--tserver_master_addrs=127.5.61.62:38651
--webserver_port=44001
--webserver_interface=127.5.61.3
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:42167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.61.3
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.3.keytab with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 20:18:17 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:17 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 20:18:17.962019 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:17.947780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:58183 (local address 127.5.61.62:38651)
0809 20:18:17.947973 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:17.947979 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:17.949264 (+ 1285us) server_negotiation.cc:407] Connection header received
0809 20:18:17.951121 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:17.951130 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:17.951220 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:17.951342 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:17.952809 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:17.953959 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:17.955514 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:17.956274 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:17.956421 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:17.961761 (+ 5340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.2:58183: BlockingRecv error: recv got EOF from 127.5.61.2:58183 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 20:18:17.962723 15895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:17.948153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.2:58183)
0809 20:18:17.949100 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:17.949153 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:17.950819 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:17.951534 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:17.951541 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:17.952098 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:17.952614 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:17.952630 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:17.954140 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:17.954144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:17.955351 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:17.955361 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:17.956477 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:17.956482 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:17.956702 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:17.957383 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:17.957404 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:17.961581 (+ 4177us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":464,"threads_started":1}
W20260809 20:18:17.963379 15892 heartbeater.cc:644] Failed to heartbeat to 127.5.61.62:38651 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.61.62:38651: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Aug 09 20:18:17 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:17 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:17.971616 15895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:17.963993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.2:49643)
0809 20:18:17.964205 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:17.964223 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:17.964331 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:17.964641 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:17.964645 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:17.964981 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:17.965105 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:17.965112 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:17.966180 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:17.966188 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:17.967052 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:17.967061 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:17.967823 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:17.967829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:17.968000 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:17.968572 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:17.968588 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:17.971268 (+ 2680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121}
W20260809 20:18:17.971900 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:17.964048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:49643 (local address 127.5.61.62:38651)
0809 20:18:17.964207 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:17.964214 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:17.964272 (+ 58us) server_negotiation.cc:407] Connection header received
0809 20:18:17.964452 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:17.964459 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:17.964521 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:17.964620 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:17.965293 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:17.966041 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:17.967193 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:17.967653 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:17.967815 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:17.971489 (+ 3674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.2:49643: BlockingRecv error: recv got EOF from 127.5.61.2:49643 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 20:18:17 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:17 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:17.980676 15895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:17.972699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.2:36965)
0809 20:18:17.972948 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:17.972966 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:17.973105 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:17.973435 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:17.973439 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:17.973732 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:17.973858 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:17.973865 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:17.975027 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:17.975036 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:17.975959 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:17.975969 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:17.976626 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:17.976631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:17.976796 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:17.977357 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:17.977373 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:17.980262 (+ 2889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140}
W20260809 20:18:17.980674 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:17.972810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:36965 (local address 127.5.61.62:38651)
0809 20:18:17.972950 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:17.972957 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:17.973047 (+ 90us) server_negotiation.cc:407] Connection header received
0809 20:18:17.973221 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:17.973225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:17.973315 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:17.973412 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:17.973984 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:17.974882 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:17.976097 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:17.976482 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:17.976605 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:17.980468 (+ 3863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.2:36965: BlockingRecv error: recv got EOF from 127.5.61.2:36965 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 20:18:17.981215 15892 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 20:18:18.285456 15896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:18.285712 15896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:18.285804 15896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:18.292917 15896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:18.293008 15896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:18.293114 15896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:18:18.302322 15896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:42167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/krb5kdc/oryx_127.5.61.3.keytab
--principal=oryx/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:38439
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=44001
--tserver_master_addrs=127.5.61.62:38651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:18.303598 15896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:18.304939 15896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:18.318130 15901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:18.318173 15902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:18.318311 15904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:18.319303 15896 server_base.cc:1034] running on GCE node
I20260809 20:18:18.320029 15896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:18.320948 15896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:18.322192 15896 hybrid_clock.cc:648] HybridClock initialized: now 1786306698322150 us; error 51 us; skew 500 ppm
I20260809 20:18:18.322513 15896 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:18:18 dist-test-slave-6k22 krb5kdc[14920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306698, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:18:18.325733 15896 init.cc:377] Logged in from keytab as oryx/127.5.61.3@KRBTEST.COM (short username oryx)
I20260809 20:18:18.327176 15896 webserver.cc:460] Webserver started at http://127.5.61.3:44001/ using document root <none> and password file <none>
I20260809 20:18:18.327943 15896 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:18.328105 15896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:18.333410 15896 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 20:18:18.336254 15911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:18.337522 15896 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:18:18.337689 15896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "8230bf6db4c9435cbf289fbead860d7a"
format_stamp: "Formatted at 2026-08-09 20:18:15 on dist-test-slave-6k22"
server_key: "199d9bfb628878da0eb69dbcec7cbf33"
server_key_iv: "b41869523b3afbe81b66369ab32b3bf3"
server_key_version: "encryptionkey@0"
I20260809 20:18:18.338654 15896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:18.358037 15896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:18.362104 15896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:18.362382 15896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:18.363334 15896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:18:18.364683 15896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:18:18.364810 15896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:18.364948 15896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:18:18.365041 15896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:18.398882 15896 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:38439
I20260809 20:18:18.398972 16033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:38439 every 8 connection(s)
I20260809 20:18:18.400406 15896 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:18:18.407701 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 15896
Aug 09 20:18:18 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:18 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 20:18:18.419370 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:18.403018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:58045 (local address 127.5.61.62:38651)
0809 20:18:18.403222 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:18.403227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:18.404531 (+ 1304us) server_negotiation.cc:407] Connection header received
0809 20:18:18.406320 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:18.406325 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:18.406394 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:18.406483 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:18.408355 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:18.409549 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:18.412025 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:18.412715 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:18.412838 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:18.419015 (+ 6177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.3:58045: BlockingRecv error: recv got EOF from 127.5.61.3:58045 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 20:18:18.419960 16036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:18.403419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.3:58045)
0809 20:18:18.404347 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:18.404400 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:18.406089 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:18.406704 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:18.406712 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:18.407288 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:18.408144 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:18.408172 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:18.409721 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:18.409726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:18.411895 (+ 2169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:18.411908 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:18.412916 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:18.412921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:18.413157 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:18.413918 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:18.413976 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:18.418768 (+ 4792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":453,"threads_started":1}
W20260809 20:18:18.420753 16034 heartbeater.cc:644] Failed to heartbeat to 127.5.61.62:38651 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.61.62:38651: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
I20260809 20:18:18.426506 16038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:18.408824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39128 (local address 127.5.61.62:38651)
0809 20:18:18.409607 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:18.409615 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:18.409634 (+ 19us) server_negotiation.cc:407] Connection header received
0809 20:18:18.409680 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:18.409684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:18.409745 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:18.409820 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:18.411805 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:18.413004 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:18.414876 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:18.415394 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:18.415526 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:18.418876 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:18.418892 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:18.418915 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:18.418958 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:18.423205 (+ 4247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:18.423976 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:18.423982 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:18.423988 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:18.424173 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:18.424604 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:18.424607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:18.424609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:18.425098 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:18.425394 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:18.425787 (+ 393us) server_negotiation.cc:299] Negotiation successful
0809 20:18:18.426080 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":666,"thread_start_us":547,"threads_started":1}
Aug 09 20:18:18 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:18 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:18.429872 16036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:18.421407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.3:49579)
0809 20:18:18.421573 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:18.421591 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:18.421680 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:18.421976 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:18.421979 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:18.422264 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:18.422395 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:18.422402 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:18.423719 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:18.423725 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:18.424895 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:18.424907 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:18.425828 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:18.425833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:18.426029 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:18.426547 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:18.426562 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:18.429662 (+ 3100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260809 20:18:18.430099 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:18.421489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:49579 (local address 127.5.61.62:38651)
0809 20:18:18.421675 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:18.421680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:18.421703 (+ 23us) server_negotiation.cc:407] Connection header received
0809 20:18:18.421759 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:18.421774 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:18.421846 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:18.421966 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:18.422531 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:18.423595 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:18.425027 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:18.425670 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:18.425794 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:18.429872 (+ 4078us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.3:49579: BlockingRecv error: recv got EOF from 127.5.61.3:49579 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 09 20:18:18 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:18 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:18.437914 16036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:18.430797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.3:60649)
0809 20:18:18.430990 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:18.431009 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:18.431097 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:18.431319 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:18.431322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:18.431604 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:18.431745 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:18.431752 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:18.432681 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:18.432684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:18.433667 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:18.433678 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:18.434326 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:18.434330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:18.434437 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:18.434941 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:18.434956 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:18.437606 (+ 2650us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 20:18:18.438179 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:18.430896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:60649 (local address 127.5.61.62:38651)
0809 20:18:18.431106 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:18.431110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:18.431125 (+ 15us) server_negotiation.cc:407] Connection header received
0809 20:18:18.431160 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:18.431163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:18.431209 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:18.431280 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:18.431876 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:18.432572 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:18.433778 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:18.434204 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:18.434322 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:18.437808 (+ 3486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.3:60649: BlockingRecv error: recv got EOF from 127.5.61.3:60649 (error 108)
Metrics: {"server-negotiator.queue_time_us":128}
W20260809 20:18:18.438195 16034 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 20:18:18 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:18 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:18.533243 16039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:18.524370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.1:59935)
0809 20:18:18.524939 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:18.524956 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:18.525113 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:18.525467 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:18.525472 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:18.525989 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:18.526188 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:18.526197 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:18.527212 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:18.527218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:18.528299 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:18.528311 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:18.529086 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:18.529092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:18.529260 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:18.529953 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:18.529970 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:18.532857 (+ 2887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":308,"threads_started":1}
W20260809 20:18:18.533511 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:18.524468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:59935 (local address 127.5.61.62:38651)
0809 20:18:18.524665 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:18.524671 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:18.525043 (+ 372us) server_negotiation.cc:407] Connection header received
0809 20:18:18.525237 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:18.525244 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:18.525313 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:18.525437 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:18.526336 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:18.527068 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:18.528450 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:18.528892 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:18.529050 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:18.533135 (+ 4085us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.1:59935: BlockingRecv error: recv got EOF from 127.5.61.1:59935 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 20:18:18 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:18 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:18.990761 16041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:18.981791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.2:56655)
0809 20:18:18.982326 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:18.982342 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:18.982510 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:18.982844 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:18.982849 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:18.983357 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:18.983586 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:18.983596 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:18.984673 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:18.984678 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:18.985647 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:18.985659 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:18.986436 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:18.986442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:18.986631 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:18.987307 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:18.987326 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:18.990438 (+ 3112us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":271,"threads_started":1}
W20260809 20:18:18.990998 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:18.981872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:56655 (local address 127.5.61.62:38651)
0809 20:18:18.982136 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:18.982144 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:18.982436 (+ 292us) server_negotiation.cc:407] Connection header received
0809 20:18:18.982636 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:18.982641 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:18.982709 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:18.982811 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:18.983737 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:18.984519 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:18.985808 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:18.986249 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:18.986408 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:18.990718 (+ 4310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.2:56655: BlockingRecv error: recv got EOF from 127.5.61.2:56655 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 09 20:18:19 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:19 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:19.447676 16042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:19.438767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.3:36123)
0809 20:18:19.439293 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:19.439310 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:19.439424 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:19.439781 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:19.439784 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:19.440259 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:19.440457 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:19.440465 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:19.441836 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:19.441839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:19.442867 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:19.442878 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:19.443787 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:19.443792 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:19.443920 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:19.444465 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:19.444482 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:19.447290 (+ 2808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":287,"threads_started":1}
W20260809 20:18:19.447806 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:19.438841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:36123 (local address 127.5.61.62:38651)
0809 20:18:19.439022 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:19.439026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:19.439388 (+ 362us) server_negotiation.cc:407] Connection header received
0809 20:18:19.439521 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:19.439524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:19.439643 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:19.439725 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:19.440590 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:19.441669 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:19.442989 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:19.443507 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:19.443684 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:19.447510 (+ 3826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.3:36123: BlockingRecv error: recv got EOF from 127.5.61.3:36123 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 20:18:19 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:19 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:19.542562 16043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:19.534165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.1:47909)
0809 20:18:19.534760 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:19.534780 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:19.534957 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:19.535352 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:19.535355 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:19.535937 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:19.536080 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:19.536090 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:19.537049 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:19.537053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:19.537948 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:19.537959 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:19.538604 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:19.538608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:19.538715 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:19.539270 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:19.539286 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:19.542107 (+ 2821us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":287,"threads_started":1}
W20260809 20:18:19.542575 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:19.534221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:47909 (local address 127.5.61.62:38651)
0809 20:18:19.534472 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:19.534479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:19.534890 (+ 411us) server_negotiation.cc:407] Connection header received
0809 20:18:19.535082 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:19.535087 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:19.535182 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:19.535306 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:19.536206 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:19.536932 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:19.538065 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:19.538469 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:19.538569 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:19.542286 (+ 3717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.1:47909: BlockingRecv error: recv got EOF from 127.5.61.1:47909 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 20:18:19 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:19 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:20.000265 16045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:19.991774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.2:42559)
0809 20:18:19.992324 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:19.992341 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:19.992468 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:19.992765 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:19.992768 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:19.993151 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:19.993377 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:19.993390 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:19.994625 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:19.994629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:19.995823 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:19.995836 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:19.996517 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:19.996521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:19.996626 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:19.997217 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:19.997234 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:20.000013 (+ 2779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":312,"threads_started":1}
W20260809 20:18:20.000407 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:19.991850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:42559 (local address 127.5.61.62:38651)
0809 20:18:19.992025 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:19.992030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:19.992415 (+ 385us) server_negotiation.cc:407] Connection header received
0809 20:18:19.992566 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:19.992569 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:19.992629 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:19.992712 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:19.993529 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:19.994459 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:19.995959 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:19.996383 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:19.996477 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:20.000199 (+ 3722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.2:42559: BlockingRecv error: recv got EOF from 127.5.61.2:42559 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 20:18:20 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:20 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:20.457491 16046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:20.448577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.3:48157)
0809 20:18:20.449145 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:20.449160 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:20.449314 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:20.449674 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:20.449679 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:20.450109 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:20.450258 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:20.450267 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:20.451388 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:20.451393 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:20.452529 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:20.452542 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:20.453586 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:20.453592 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:20.453779 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:20.454505 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:20.454523 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:20.457254 (+ 2731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":459,"threads_started":1}
W20260809 20:18:20.457720 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:20.448769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:48157 (local address 127.5.61.62:38651)
0809 20:18:20.448999 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:20.449005 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:20.449242 (+ 237us) server_negotiation.cc:407] Connection header received
0809 20:18:20.449459 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:20.449464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:20.449533 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:20.449633 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:20.450413 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:20.451178 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:20.452707 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:20.453412 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:20.453550 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:20.457439 (+ 3889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.3:48157: BlockingRecv error: recv got EOF from 127.5.61.3:48157 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 09 20:18:20 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:20 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:20.551963 16047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:20.543450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.1:53915)
0809 20:18:20.543952 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:20.543969 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:20.544085 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:20.544377 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:20.544380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:20.544721 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:20.544904 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:20.544913 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:20.545914 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:20.545917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:20.547219 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:20.547235 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:20.548044 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:20.548048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:20.548171 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:20.548762 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:20.548784 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:20.551557 (+ 2773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":298,"threads_started":1}
W20260809 20:18:20.552019 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:20.543589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:53915 (local address 127.5.61.62:38651)
0809 20:18:20.543848 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:20.543854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:20.544047 (+ 193us) server_negotiation.cc:407] Connection header received
0809 20:18:20.544179 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:20.544182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:20.544251 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:20.544331 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:20.545027 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:20.545754 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:20.547365 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:20.547906 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:20.547998 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:20.551770 (+ 3772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.1:53915: BlockingRecv error: recv got EOF from 127.5.61.1:53915 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 09 20:18:21 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:21 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:21.010092 16048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:21.001247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.2:40815)
0809 20:18:21.001816 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:21.001831 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:21.001995 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:21.002330 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:21.002334 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:21.002730 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:21.002919 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:21.002928 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:21.004231 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:21.004236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:21.005415 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:21.005428 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:21.006152 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:21.006156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:21.006282 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:21.006951 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:21.006968 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:21.009839 (+ 2871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":314,"threads_started":1}
W20260809 20:18:21.010283 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:21.001431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:40815 (local address 127.5.61.62:38651)
0809 20:18:21.001663 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:21.001670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:21.001917 (+ 247us) server_negotiation.cc:407] Connection header received
0809 20:18:21.002134 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:21.002138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:21.002198 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:21.002297 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:21.003124 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:21.004075 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:21.005570 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:21.006027 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:21.006125 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:21.010040 (+ 3915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.2:40815: BlockingRecv error: recv got EOF from 127.5.61.2:40815 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 20:18:21 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:21 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:21.466638 16049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:21.458395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.3:36891)
0809 20:18:21.458954 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:21.458969 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:21.459084 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:21.459404 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:21.459407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:21.459775 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:21.459942 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:21.459951 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:21.460925 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:21.460929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:21.461846 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:21.461856 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:21.462487 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:21.462491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:21.462611 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:21.463171 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:21.463188 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:21.466391 (+ 3203us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":356,"threads_started":1}
W20260809 20:18:21.466856 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:21.458480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:36891 (local address 127.5.61.62:38651)
0809 20:18:21.458643 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:21.458648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:21.459057 (+ 409us) server_negotiation.cc:407] Connection header received
0809 20:18:21.459193 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:21.459197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:21.459268 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:21.459349 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:21.460070 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:21.460798 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:21.461965 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:21.462366 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:21.462455 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:21.466637 (+ 4182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.3:36891: BlockingRecv error: recv got EOF from 127.5.61.3:36891 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 20:18:21 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:21 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:21.561421 16050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:21.552786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.1:55949)
0809 20:18:21.553326 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:21.553343 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:21.553490 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:21.553773 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:21.553776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:21.554134 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:21.554320 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:21.554329 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:21.555268 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:21.555271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:21.556318 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:21.556330 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:21.556939 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:21.556942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:21.557067 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:21.557814 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:21.557862 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:21.561058 (+ 3196us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":300,"threads_started":1}
W20260809 20:18:21.561559 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:21.552886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:55949 (local address 127.5.61.62:38651)
0809 20:18:21.553088 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:21.553092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:21.553450 (+ 358us) server_negotiation.cc:407] Connection header received
0809 20:18:21.553585 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:21.553588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:21.553646 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:21.553725 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:21.554441 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:21.555152 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:21.556439 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:21.556818 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:21.556912 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:21.561260 (+ 4348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.1:55949: BlockingRecv error: recv got EOF from 127.5.61.1:55949 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 09 20:18:22 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:22 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:22.020272 16052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:22.011126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.2:36935)
0809 20:18:22.011778 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:22.011794 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:22.011959 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:22.012298 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:22.012302 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:22.012705 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:22.012859 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:22.012868 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:22.013917 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:22.013925 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:22.014871 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:22.014882 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:22.015778 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:22.015784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:22.015985 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:22.016799 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:22.016824 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:22.020051 (+ 3227us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":317,"threads_started":1}
W20260809 20:18:22.020430 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:22.011195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:36935 (local address 127.5.61.62:38651)
0809 20:18:22.011456 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:22.011463 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:22.011883 (+ 420us) server_negotiation.cc:407] Connection header received
0809 20:18:22.012080 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:22.012085 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:22.012158 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:22.012266 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:22.013000 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:22.013750 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:22.015009 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:22.015625 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:22.015755 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:22.020225 (+ 4470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.2:36935: BlockingRecv error: recv got EOF from 127.5.61.2:36935 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 20:18:22 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:22 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:22.480472 16053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:22.467711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.3:40865)
0809 20:18:22.468212 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:22.468228 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:22.468397 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:22.468747 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:22.468752 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:22.469153 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:22.469301 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:22.469310 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:22.470379 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:22.470390 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:22.471333 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:22.471345 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:22.472143 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:22.472148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:22.472336 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:22.477212 (+ 4876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:22.477230 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:22.480245 (+ 3015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":285,"threads_started":1}
W20260809 20:18:22.480732 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:22.467745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:40865 (local address 127.5.61.62:38651)
0809 20:18:22.467962 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:22.467970 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:22.468314 (+ 344us) server_negotiation.cc:407] Connection header received
0809 20:18:22.468518 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:22.468523 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:22.468603 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:22.468708 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:22.469450 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:22.470233 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:22.471484 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:22.471990 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:22.472132 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:22.480454 (+ 8322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.3:40865: BlockingRecv error: recv got EOF from 127.5.61.3:40865 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 20:18:22 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:22 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:22.570473 16054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:22.562411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.1:33603)
0809 20:18:22.562961 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:22.562978 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:22.563086 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:22.563356 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:22.563359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:22.563797 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:22.563969 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:22.563978 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:22.564962 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:22.564965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:22.565873 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:22.565884 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:22.566512 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:22.566516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:22.566633 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:22.567216 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:22.567233 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:22.570152 (+ 2919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":277,"threads_started":1}
W20260809 20:18:22.570572 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:22.562499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:33603 (local address 127.5.61.62:38651)
0809 20:18:22.562699 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:22.562703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:22.563053 (+ 350us) server_negotiation.cc:407] Connection header received
0809 20:18:22.563169 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:22.563172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:22.563231 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:22.563309 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:22.564101 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:22.564839 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:22.565992 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:22.566391 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:22.566484 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:22.570302 (+ 3818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.1:33603: BlockingRecv error: recv got EOF from 127.5.61.1:33603 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 20:18:23 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:23 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:23.029342 16055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:23.021161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.2:34519)
0809 20:18:23.021695 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:23.021710 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:23.021818 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:23.022093 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:23.022096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:23.022526 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:23.022714 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:23.022723 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:23.023765 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:23.023770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:23.024647 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:23.024657 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:23.025261 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:23.025265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:23.025390 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:23.025958 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:23.025974 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:23.029066 (+ 3092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":261,"threads_started":1}
W20260809 20:18:23.029480 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:23.021371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:34519 (local address 127.5.61.62:38651)
0809 20:18:23.021610 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:23.021616 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:23.021794 (+ 178us) server_negotiation.cc:407] Connection header received
0809 20:18:23.021911 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:23.021914 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:23.021971 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:23.022048 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:23.022845 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:23.023643 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:23.024770 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:23.025146 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:23.025233 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:23.029246 (+ 4013us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.2:34519: BlockingRecv error: recv got EOF from 127.5.61.2:34519 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 09 20:18:23 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:23 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:23.491040 16056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:23.481503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.3:40543)
0809 20:18:23.482042 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:23.482058 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:23.482169 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:23.482490 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:23.482497 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:23.483012 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:23.483254 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:23.483267 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:23.484353 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:23.484358 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:23.485843 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:23.485858 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:23.486647 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:23.486652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:23.486821 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:23.487552 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:23.487600 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:23.490810 (+ 3210us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":272,"threads_started":1}
W20260809 20:18:23.491266 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:23.481554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:40543 (local address 127.5.61.62:38651)
0809 20:18:23.481723 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:23.481727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:23.482131 (+ 404us) server_negotiation.cc:407] Connection header received
0809 20:18:23.482264 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:23.482267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:23.482327 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:23.482407 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:23.483405 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:23.484203 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:23.485983 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:23.486488 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:23.486582 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:23.491006 (+ 4424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.3:40543: BlockingRecv error: recv got EOF from 127.5.61.3:40543 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 20:18:23 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:23 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:23.579164 16058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:23.571446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.1:60685)
0809 20:18:23.571994 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:23.572010 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:23.572125 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:23.572436 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:23.572439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:23.572841 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:23.572997 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:23.573005 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:23.574004 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:23.574007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:23.574875 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:23.574885 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:23.575546 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:23.575551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:23.575726 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:23.576279 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:23.576296 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:23.578953 (+ 2657us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":348,"threads_started":1}
W20260809 20:18:23.579401 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:23.571508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:60685 (local address 127.5.61.62:38651)
0809 20:18:23.571762 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:23.571770 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:23.572094 (+ 324us) server_negotiation.cc:407] Connection header received
0809 20:18:23.572221 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:23.572224 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:23.572299 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:23.572377 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:23.573119 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:23.573886 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:23.574994 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:23.575409 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:23.575497 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:23.579149 (+ 3652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.1:60685: BlockingRecv error: recv got EOF from 127.5.61.1:60685 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 20:18:24 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:24 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:24.039374 16059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:24.030340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.2:46773)
0809 20:18:24.030920 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:24.030936 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:24.031095 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:24.031460 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:24.031465 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:24.031959 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:24.032170 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:24.032180 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:24.033588 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:24.033593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:24.034640 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:24.034652 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:24.035302 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:24.035306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:24.035430 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:24.036225 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:24.036245 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:24.039145 (+ 2900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":343,"threads_started":1}
W20260809 20:18:24.039724 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:24.030393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:46773 (local address 127.5.61.62:38651)
0809 20:18:24.030654 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:24.030663 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:24.031024 (+ 361us) server_negotiation.cc:407] Connection header received
0809 20:18:24.031217 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:24.031222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:24.031313 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:24.031426 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:24.032343 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:24.033414 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:24.034769 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:24.035169 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:24.035265 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:24.039384 (+ 4119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.2:46773: BlockingRecv error: recv got EOF from 127.5.61.2:46773 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 20:18:24 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:24 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:24.500456 16060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:24.491981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.3:45117)
0809 20:18:24.492499 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:24.492515 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:24.492664 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:24.493015 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:24.493020 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:24.493453 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:24.493603 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:24.493612 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:24.494824 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:24.494830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:24.495961 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:24.495977 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:24.496812 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:24.496817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:24.496935 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:24.497499 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:24.497516 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:24.500245 (+ 2729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":287,"threads_started":1}
W20260809 20:18:24.500751 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:24.492063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:45117 (local address 127.5.61.62:38651)
0809 20:18:24.492304 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:24.492312 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:24.492602 (+ 290us) server_negotiation.cc:407] Connection header received
0809 20:18:24.492805 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:24.492810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:24.492879 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:24.492981 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:24.493756 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:24.494683 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:24.496119 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:24.496657 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:24.496762 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:24.500465 (+ 3703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.3:45117: BlockingRecv error: recv got EOF from 127.5.61.3:45117 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 09 20:18:24 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:24 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:24.588229 16061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:24.580115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.1:50077)
0809 20:18:24.580784 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:24.580800 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:24.580918 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:24.581217 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:24.581220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:24.581628 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:24.581799 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:24.581808 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:24.582745 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:24.582748 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:24.583676 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:24.583687 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:24.584346 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:24.584350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:24.584469 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:24.585023 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:24.585040 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:24.587855 (+ 2815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":301,"threads_started":1}
W20260809 20:18:24.588274 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:24.580181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:50077 (local address 127.5.61.62:38651)
0809 20:18:24.580361 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:24.580365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:24.580870 (+ 505us) server_negotiation.cc:407] Connection header received
0809 20:18:24.581012 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:24.581016 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:24.581074 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:24.581154 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:24.581920 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:24.582618 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:24.583801 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:24.584214 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:24.584311 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:24.588040 (+ 3729us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.1:50077: BlockingRecv error: recv got EOF from 127.5.61.1:50077 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 20:18:25 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:25 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:25.048557 16062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:25.040337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.2:48037)
0809 20:18:25.040933 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:25.040949 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:25.041076 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:25.041363 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:25.041368 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:25.041840 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:25.042029 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:25.042037 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:25.043163 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:25.043167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:25.044088 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:25.044098 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:25.044729 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:25.044733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:25.044835 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:25.045387 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:25.045404 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:25.048307 (+ 2903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":265,"threads_started":1}
W20260809 20:18:25.048799 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:25.040416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:48037 (local address 127.5.61.62:38651)
0809 20:18:25.040707 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:25.040713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:25.041042 (+ 329us) server_negotiation.cc:407] Connection header received
0809 20:18:25.041169 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:25.041172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:25.041233 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:25.041321 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:25.042171 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:25.043016 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:25.044202 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:25.044603 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:25.044695 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:25.048460 (+ 3765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.2:48037: BlockingRecv error: recv got EOF from 127.5.61.2:48037 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Aug 09 20:18:25 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:25 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:25.509531 16063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:25.501429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.3:47591)
0809 20:18:25.502020 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:25.502037 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:25.502148 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:25.502444 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:25.502447 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:25.502842 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:25.503020 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:25.503028 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:25.504065 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:25.504069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:25.504976 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:25.504987 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:25.505719 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:25.505723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:25.505832 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:25.506396 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:25.506413 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:25.509290 (+ 2877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":239,"threads_started":1}
W20260809 20:18:25.509666 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:25.501475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:47591 (local address 127.5.61.62:38651)
0809 20:18:25.501681 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:25.501690 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:25.502115 (+ 425us) server_negotiation.cc:407] Connection header received
0809 20:18:25.502250 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:25.502254 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:25.502315 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:25.502395 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:25.503133 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:25.503939 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:25.505128 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:25.505591 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:25.505686 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:25.509480 (+ 3794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.3:47591: BlockingRecv error: recv got EOF from 127.5.61.3:47591 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 09 20:18:25 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:25 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:25.598152 16064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:25.589266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.1:53291)
0809 20:18:25.589897 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:25.589921 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:25.590071 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:25.590433 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:25.590437 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:25.590914 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:25.591138 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:25.591148 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:25.592234 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:25.592240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:25.593149 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:25.593160 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:25.594101 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:25.594107 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:25.594293 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:25.595069 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:25.595095 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:25.597931 (+ 2836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":299,"threads_started":1}
W20260809 20:18:25.598420 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:25.589473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:53291 (local address 127.5.61.62:38651)
0809 20:18:25.589716 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:25.589722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:25.590019 (+ 297us) server_negotiation.cc:407] Connection header received
0809 20:18:25.590206 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:25.590213 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:25.590281 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:25.590406 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:25.591279 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:25.592092 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:25.593291 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:25.593916 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:25.594042 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:25.598118 (+ 4076us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.1:53291: BlockingRecv error: recv got EOF from 127.5.61.1:53291 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 09 20:18:26 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:26 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:26.058379 16065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:26.049536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.2:34327)
0809 20:18:26.050086 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:26.050102 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:26.050276 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:26.050619 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:26.050623 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:26.051062 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:26.051218 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:26.051226 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:26.052933 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:26.052939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:26.054012 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:26.054024 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:26.054732 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:26.054736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:26.054849 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:26.055466 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:26.055483 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:26.058114 (+ 2631us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":316,"threads_started":1}
W20260809 20:18:26.058532 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:26.049638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:34327 (local address 127.5.61.62:38651)
0809 20:18:26.049851 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:26.049859 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:26.050186 (+ 327us) server_negotiation.cc:407] Connection header received
0809 20:18:26.050403 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:26.050414 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:26.050479 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:26.050583 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:26.051372 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:26.052745 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:26.054139 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:26.054577 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:26.054671 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:26.058314 (+ 3643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.2:34327: BlockingRecv error: recv got EOF from 127.5.61.2:34327 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 20:18:26 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:26 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:26.518824 16066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:26.510554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.3:37333)
0809 20:18:26.511176 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:26.511197 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:26.511362 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:26.511783 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:26.511787 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:26.512331 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:26.512500 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:26.512510 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:26.513476 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:26.513479 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:26.514308 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:26.514318 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:26.515051 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:26.515056 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:26.515171 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:26.515870 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:26.515886 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:26.518552 (+ 2666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":350,"threads_started":1}
W20260809 20:18:26.519127 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:26.510641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:37333 (local address 127.5.61.62:38651)
0809 20:18:26.510941 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:26.510948 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:26.511293 (+ 345us) server_negotiation.cc:407] Connection header received
0809 20:18:26.511504 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:26.511509 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:26.511627 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:26.511728 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:26.512633 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:26.513355 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:26.514474 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:26.514901 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:26.514999 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:26.518860 (+ 3861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.3:37333: BlockingRecv error: recv got EOF from 127.5.61.3:37333 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 09 20:18:26 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:26 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:26.607959 16068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:26.599070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.1:49161)
0809 20:18:26.599643 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:26.599659 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:26.599815 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:26.600179 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:26.600183 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:26.600619 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:26.600785 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:26.600793 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:26.601874 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:26.601884 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:26.602806 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:26.602816 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:26.603600 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:26.603607 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:26.603779 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:26.604454 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:26.604489 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:26.607599 (+ 3110us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":293,"threads_started":1}
W20260809 20:18:26.608034 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:26.599161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:49161 (local address 127.5.61.62:38651)
0809 20:18:26.599440 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:26.599447 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:26.599748 (+ 301us) server_negotiation.cc:407] Connection header received
0809 20:18:26.599946 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:26.599951 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:26.600041 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:26.600143 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:26.600935 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:26.601721 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:26.602957 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:26.603412 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:26.603544 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:26.607804 (+ 4260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.1:49161: BlockingRecv error: recv got EOF from 127.5.61.1:49161 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 09 20:18:27 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:27 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:27.067910 16069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:27.059372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.2:36375)
0809 20:18:27.059937 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:27.059954 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:27.060069 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:27.060356 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:27.060360 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:27.060744 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:27.060917 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:27.060924 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:27.062027 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:27.062033 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:27.063525 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:27.063538 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:27.064228 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:27.064232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:27.064353 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:27.064898 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:27.064913 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:27.067659 (+ 2746us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":332,"threads_started":1}
W20260809 20:18:27.068089 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:27.059402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:36375 (local address 127.5.61.62:38651)
0809 20:18:27.059653 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:27.059662 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:27.060042 (+ 380us) server_negotiation.cc:407] Connection header received
0809 20:18:27.060169 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:27.060173 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:27.060233 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:27.060313 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:27.061040 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:27.061846 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:27.063695 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:27.064103 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:27.064196 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:27.067868 (+ 3672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.2:36375: BlockingRecv error: recv got EOF from 127.5.61.2:36375 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 09 20:18:27 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:27 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:27.532876 16070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:27.519893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.3:49067)
0809 20:18:27.520370 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:27.520385 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:27.520535 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:27.520889 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:27.520894 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:27.521335 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:27.521507 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:27.521516 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:27.522512 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:27.522518 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:27.523457 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:27.523498 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:27.524488 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:27.524494 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:27.524682 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:27.529744 (+ 5062us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:27.529762 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:27.532628 (+ 2866us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":241,"threads_started":1}
W20260809 20:18:27.533059 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:27.519918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:49067 (local address 127.5.61.62:38651)
0809 20:18:27.520116 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:27.520124 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:27.520475 (+ 351us) server_negotiation.cc:407] Connection header received
0809 20:18:27.520670 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:27.520676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:27.520752 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:27.520854 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:27.521665 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:27.522376 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:27.523671 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:27.524302 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:27.524476 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:27.532832 (+ 8356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.3:49067: BlockingRecv error: recv got EOF from 127.5.61.3:49067 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 20:18:27 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:27 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:27.617498 16071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:27.608914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.1:48037)
0809 20:18:27.609382 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:27.609398 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:27.609550 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:27.609902 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:27.609907 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:27.610321 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:27.610473 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:27.610482 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:27.611538 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:27.611546 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:27.612529 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:27.612541 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:27.613327 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:27.613333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:27.613529 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:27.614166 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:27.614183 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:27.617177 (+ 2994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":263,"threads_started":1}
W20260809 20:18:27.617902 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:27.608968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:48037 (local address 127.5.61.62:38651)
0809 20:18:27.609167 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:27.609176 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:27.609482 (+ 306us) server_negotiation.cc:407] Connection header received
0809 20:18:27.609672 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:27.609681 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:27.609757 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:27.609861 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:27.610619 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:27.611391 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:27.612689 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:27.613148 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:27.613295 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:27.617448 (+ 4153us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.1:48037: BlockingRecv error: recv got EOF from 127.5.61.1:48037 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 20:18:28 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:28 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:28.077170 16072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:28.068897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.2:56613)
0809 20:18:28.069406 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:28.069421 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:28.069577 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:28.069928 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:28.069932 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:28.070327 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:28.070468 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:28.070475 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:28.071551 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:28.071587 (+ 36us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:28.072509 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:28.072535 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:28.073288 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:28.073293 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:28.073475 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:28.074034 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:28.074051 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:28.076953 (+ 2902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":280,"threads_started":1}
W20260809 20:18:28.077401 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:28.068980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:56613 (local address 127.5.61.62:38651)
0809 20:18:28.069213 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:28.069222 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:28.069510 (+ 288us) server_negotiation.cc:407] Connection header received
0809 20:18:28.069723 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:28.069728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:28.069795 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:28.069890 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:28.070637 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:28.071389 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:28.072671 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:28.073124 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:28.073278 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:28.077151 (+ 3873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.2:56613: BlockingRecv error: recv got EOF from 127.5.61.2:56613 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 09 20:18:28 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:28 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:28.542431 16073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:28.533832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.3:46849)
0809 20:18:28.534396 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:28.534417 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:28.534576 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:28.534926 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:28.534930 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:28.535326 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:28.535476 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:28.535484 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:28.536537 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:28.536544 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:28.537464 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:28.537475 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:28.538176 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:28.538182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:28.538362 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:28.539075 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:28.539092 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:28.542091 (+ 2999us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":279,"threads_started":1}
W20260809 20:18:28.542533 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:28.534020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:46849 (local address 127.5.61.62:38651)
0809 20:18:28.534293 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:28.534300 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:28.534501 (+ 201us) server_negotiation.cc:407] Connection header received
0809 20:18:28.534707 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:28.534712 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:28.534789 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:28.534890 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:28.535641 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:28.536393 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:28.537606 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:28.538028 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:28.538162 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:28.542301 (+ 4139us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.3:46849: BlockingRecv error: recv got EOF from 127.5.61.3:46849 (error 108)
Metrics: {"server-negotiator.queue_time_us":134}
Aug 09 20:18:28 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:28 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:28.627372 16074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:28.618486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.1:41711)
0809 20:18:28.619003 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:28.619019 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:28.619133 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:28.619445 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:28.619448 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:28.619971 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:28.620160 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:28.620168 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:28.621157 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:28.621160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:28.622035 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:28.622044 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:28.623015 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:28.623040 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:28.623212 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:28.623991 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:28.624017 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:28.627127 (+ 3110us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":259,"threads_started":1}
W20260809 20:18:28.627553 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:28.618549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:41711 (local address 127.5.61.62:38651)
0809 20:18:28.618808 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:28.618814 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:28.619120 (+ 306us) server_negotiation.cc:407] Connection header received
0809 20:18:28.619241 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:28.619246 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:28.619322 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:28.619402 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:28.620280 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:28.621042 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:28.622194 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:28.622860 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:28.623001 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:28.627316 (+ 4315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.1:41711: BlockingRecv error: recv got EOF from 127.5.61.1:41711 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 20:18:29 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:29 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:29.087615 16075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:29.078316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.2:47887)
0809 20:18:29.078876 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:29.078891 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:29.079074 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:29.079403 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:29.079408 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:29.079931 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:29.080118 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:29.080127 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:29.081148 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:29.081155 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:29.082070 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:29.082081 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:29.083289 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:29.083298 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:29.083493 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:29.084314 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:29.084344 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:29.087207 (+ 2863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":346,"threads_started":1}
W20260809 20:18:29.087741 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:29.078425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:47887 (local address 127.5.61.62:38651)
0809 20:18:29.078651 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:29.078660 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:29.078989 (+ 329us) server_negotiation.cc:407] Connection header received
0809 20:18:29.079202 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:29.079206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:29.079268 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:29.079369 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:29.080272 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:29.081012 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:29.082251 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:29.083090 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:29.083233 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:29.087433 (+ 4200us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.2:47887: BlockingRecv error: recv got EOF from 127.5.61.2:47887 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 09 20:18:29 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:29 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:29.551803 16076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:29.543498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.3:47015)
0809 20:18:29.544074 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:29.544089 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:29.544200 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:29.544492 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:29.544495 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:29.544898 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:29.545082 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:29.545090 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:29.546027 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:29.546030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:29.546928 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:29.546943 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:29.547727 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:29.547733 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:29.547933 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:29.548754 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:29.548785 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:29.551505 (+ 2720us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":336,"threads_started":1}
W20260809 20:18:29.551870 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:29.543606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:47015 (local address 127.5.61.62:38651)
0809 20:18:29.543776 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:29.543780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:29.544156 (+ 376us) server_negotiation.cc:407] Connection header received
0809 20:18:29.544295 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:29.544298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:29.544356 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:29.544437 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:29.545197 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:29.545923 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:29.547070 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:29.547548 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:29.547683 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:29.551693 (+ 4010us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.3:47015: BlockingRecv error: recv got EOF from 127.5.61.3:47015 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 20:18:29 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:29 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:29.637727 16077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:29.628332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.1:37599)
0809 20:18:29.628945 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:29.628960 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:29.629138 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:29.629512 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:29.629517 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:29.629970 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:29.630133 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:29.630142 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:29.631205 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:29.631210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:29.632597 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:29.632612 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:29.633463 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:29.633469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:29.633663 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:29.634387 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:29.634405 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:29.637389 (+ 2984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":318,"threads_started":1}
W20260809 20:18:29.637843 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:29.628558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:37599 (local address 127.5.61.62:38651)
0809 20:18:29.628821 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:29.628827 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:29.629046 (+ 219us) server_negotiation.cc:407] Connection header received
0809 20:18:29.629273 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:29.629280 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:29.629369 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:29.629475 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:29.630282 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:29.631057 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:29.632719 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:29.633307 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:29.633441 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:29.637584 (+ 4143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.1:37599: BlockingRecv error: recv got EOF from 127.5.61.1:37599 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 09 20:18:30 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:30 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:30.096654 16079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:30.088722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.2:48715)
0809 20:18:30.089283 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:30.089300 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:30.089428 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:30.089770 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:30.089773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:30.090156 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:30.090311 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:30.090319 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:30.091357 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:30.091368 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:30.092293 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:30.092303 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:30.092920 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:30.092924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:30.093047 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:30.093635 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:30.093651 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:30.096427 (+ 2776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":294,"threads_started":1}
W20260809 20:18:30.096819 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:30.088741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:48715 (local address 127.5.61.62:38651)
0809 20:18:30.088953 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:30.088963 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:30.089381 (+ 418us) server_negotiation.cc:407] Connection header received
0809 20:18:30.089512 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:30.089515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:30.089612 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:30.089737 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:30.090444 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:30.091210 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:30.092407 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:30.092801 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:30.092892 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:30.096608 (+ 3716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.2:48715: BlockingRecv error: recv got EOF from 127.5.61.2:48715 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 20:18:30 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:30 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:30.560539 16080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:30.552767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.3:58395)
0809 20:18:30.553350 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:30.553366 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:30.553478 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:30.553764 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:30.553767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:30.554160 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:30.554318 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:30.554327 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:30.555286 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:30.555289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:30.556221 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:30.556233 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:30.556900 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:30.556904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:30.557036 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:30.557614 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:30.557630 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:30.560312 (+ 2682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":326,"threads_started":1}
W20260809 20:18:30.560734 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:30.552803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:58395 (local address 127.5.61.62:38651)
0809 20:18:30.552963 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:30.552968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:30.553437 (+ 469us) server_negotiation.cc:407] Connection header received
0809 20:18:30.553570 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:30.553574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:30.553632 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:30.553713 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:30.554435 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:30.555161 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:30.556338 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:30.556776 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:30.556866 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:30.560512 (+ 3646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.3:58395: BlockingRecv error: recv got EOF from 127.5.61.3:58395 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 20:18:30 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:30 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:30.647497 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:30.638739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.1:35625)
0809 20:18:30.639279 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:30.639294 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:30.639451 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:30.639839 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:30.639844 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:30.640287 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:30.640477 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:30.640486 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:30.641539 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:30.641543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:30.642524 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:30.642536 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:30.643343 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:30.643349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:30.643519 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:30.644207 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:30.644230 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:30.647238 (+ 3008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":292,"threads_started":1}
W20260809 20:18:30.647683 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:30.638833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:35625 (local address 127.5.61.62:38651)
0809 20:18:30.639061 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:30.639069 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:30.639384 (+ 315us) server_negotiation.cc:407] Connection header received
0809 20:18:30.639595 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:30.639600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:30.639686 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:30.639805 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:30.640626 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:30.641370 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:30.642679 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:30.643182 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:30.643321 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:30.647416 (+ 4095us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.1:35625: BlockingRecv error: recv got EOF from 127.5.61.1:35625 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 09 20:18:31 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:31 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:31.106683 16082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:31.097641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.2:43771)
0809 20:18:31.098327 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:31.098342 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:31.098508 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:31.098870 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:31.098875 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:31.099304 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:31.099493 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:31.099501 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:31.100637 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:31.100642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:31.101609 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:31.101620 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:31.102396 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:31.102402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:31.102590 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:31.103180 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:31.103197 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:31.106345 (+ 3148us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":391,"threads_started":1}
W20260809 20:18:31.106865 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:31.097827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:43771 (local address 127.5.61.62:38651)
0809 20:18:31.098099 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:31.098108 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:31.098436 (+ 328us) server_negotiation.cc:407] Connection header received
0809 20:18:31.098628 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:31.098633 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:31.098729 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:31.098833 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:31.099700 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:31.100495 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:31.101758 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:31.102210 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:31.102360 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:31.106613 (+ 4253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.2:43771: BlockingRecv error: recv got EOF from 127.5.61.2:43771 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
Aug 09 20:18:31 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:31 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:31.570389 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:31.561518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.3:46309)
0809 20:18:31.562116 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:31.562133 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:31.562296 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:31.562661 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:31.562668 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:31.563109 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:31.563290 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:31.563299 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:31.564416 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:31.564423 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:31.565345 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:31.565357 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:31.566116 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:31.566124 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:31.566296 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:31.567055 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:31.567078 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:31.570056 (+ 2978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":323,"threads_started":1}
W20260809 20:18:31.570513 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:31.561661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:46309 (local address 127.5.61.62:38651)
0809 20:18:31.561910 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:31.561919 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:31.562224 (+ 305us) server_negotiation.cc:407] Connection header received
0809 20:18:31.562425 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:31.562430 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:31.562508 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:31.562616 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:31.563443 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:31.564275 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:31.565498 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:31.565933 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:31.566076 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:31.570281 (+ 4205us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.3:46309: BlockingRecv error: recv got EOF from 127.5.61.3:46309 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 09 20:18:31 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:31 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:31.657186 16084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:31.648613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.1:54705)
0809 20:18:31.649186 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:31.649201 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:31.649381 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:31.649773 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:31.649778 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:31.650158 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:31.650326 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:31.650335 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:31.651476 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:31.651482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:31.652677 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:31.652690 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:31.653388 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:31.653393 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:31.653572 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:31.654183 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:31.654200 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:31.656951 (+ 2751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":304,"threads_started":1}
W20260809 20:18:31.657383 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:31.648731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:54705 (local address 127.5.61.62:38651)
0809 20:18:31.649035 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:31.649042 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:31.649290 (+ 248us) server_negotiation.cc:407] Connection header received
0809 20:18:31.649527 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:31.649531 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:31.649608 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:31.649716 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:31.650489 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:31.651327 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:31.652828 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:31.653233 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:31.653377 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:31.657156 (+ 3779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.1:54705: BlockingRecv error: recv got EOF from 127.5.61.1:54705 (error 108)
Metrics: {"server-negotiator.queue_time_us":154}
Aug 09 20:18:32 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:32 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:32.116402 16085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:32.107816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.2:50697)
0809 20:18:32.108269 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:32.108285 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:32.108401 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:32.108725 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:32.108731 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:32.109290 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:32.109532 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:32.109553 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:32.110680 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:32.110686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:32.111639 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:32.111650 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:32.112381 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:32.112387 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:32.112557 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:32.113154 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:32.113170 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:32.116162 (+ 2992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":234,"threads_started":1}
W20260809 20:18:32.116540 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:32.107889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:50697 (local address 127.5.61.62:38651)
0809 20:18:32.108116 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:32.108124 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:32.108371 (+ 247us) server_negotiation.cc:407] Connection header received
0809 20:18:32.108502 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:32.108505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:32.108568 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:32.108684 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:32.109704 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:32.110519 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:32.111779 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:32.112217 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:32.112349 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:32.116347 (+ 3998us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.2:50697: BlockingRecv error: recv got EOF from 127.5.61.2:50697 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 20:18:32 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:32 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.3@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:32.584033 16086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:32.571412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.3:58463)
0809 20:18:32.571958 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:32.571973 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:32.572086 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:32.572382 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:32.572384 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:32.572755 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:32.572920 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:32.572929 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:32.573882 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:32.573886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:32.574744 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:32.574754 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:32.575380 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:32.575384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:32.575498 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:32.580716 (+ 5218us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:32.580734 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:32.583778 (+ 3044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":313,"threads_started":1}
W20260809 20:18:32.584241 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:32.571482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:58463 (local address 127.5.61.62:38651)
0809 20:18:32.571713 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:32.571718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:32.572047 (+ 329us) server_negotiation.cc:407] Connection header received
0809 20:18:32.572181 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:32.572184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:32.572246 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:32.572326 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:32.573040 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:32.573752 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:32.574860 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:32.575256 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:32.575342 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:32.584024 (+ 8682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.3:58463: BlockingRecv error: recv got EOF from 127.5.61.3:58463 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 20:18:32 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:32 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.1@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:32.666859 16087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:32.658315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.1:41891)
0809 20:18:32.658922 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:32.658938 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:32.659049 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:32.659363 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:32.659367 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:32.659809 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:32.660007 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:32.660016 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:32.661105 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:32.661109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:32.662032 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:32.662043 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:32.662668 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:32.662672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:32.662776 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:32.663513 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:32.663536 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:32.666617 (+ 3081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":347,"threads_started":1}
W20260809 20:18:32.667002 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:32.658445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:41891 (local address 127.5.61.62:38651)
0809 20:18:32.658618 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:32.658622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:32.659014 (+ 392us) server_negotiation.cc:407] Connection header received
0809 20:18:32.659154 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:32.659157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:32.659233 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:32.659319 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:32.660168 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:32.660976 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:32.662151 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:32.662536 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:32.662624 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:32.666797 (+ 4173us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.1:41891: BlockingRecv error: recv got EOF from 127.5.61.1:41891 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 20:18:33 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 20:18:33 dist-test-slave-6k22 krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.61.2@KRBTEST.COM for oryx/127.5.61.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 20:18:33.126221 16088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:33.117548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38651 (local address 127.5.61.2:52615)
0809 20:18:33.118081 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:33.118097 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:33.118200 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:33.118506 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:33.118510 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:33.118957 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:33.119134 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:33.119142 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:33.120168 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:33.120171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:33.121056 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:33.121067 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:33.121936 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:33.121939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:33.122050 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:33.122763 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:33.122788 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:33.125863 (+ 3075us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.61.62:38651: Server oryx/127.5.61.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":279,"threads_started":1}
W20260809 20:18:33.126315 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:33.117604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:52615 (local address 127.5.61.62:38651)
0809 20:18:33.117753 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:33.117757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:33.118160 (+ 403us) server_negotiation.cc:407] Connection header received
0809 20:18:33.118294 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:33.118297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:33.118355 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:33.118460 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:33.119265 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:33.120038 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:33.121170 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:33.121811 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:33.121903 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:33.126067 (+ 4164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.61.2:52615: BlockingRecv error: recv got EOF from 127.5.61.2:52615 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 20:18:33.416132 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 15585
I20260809 20:18:33.428141 16090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:33.417160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45332 (local address 127.5.61.1:46095)
0809 20:18:33.417716 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:33.417732 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:33.417761 (+ 29us) server_negotiation.cc:407] Connection header received
0809 20:18:33.417827 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:33.417833 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:33.418063 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:33.418219 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:33.419762 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:33.420749 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:33.421888 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:33.422403 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:33.422499 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:33.424923 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:33.424931 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:33.424943 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:33.424975 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:33.427532 (+ 2557us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.61.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 20:18:33.427871 (+ 339us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45332: Request ticket server kudu/127.5.61.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":259,"threads_started":1}
W20260809 20:18:33.428248 16090 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45332: Request ticket server kudu/127.5.61.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 20:18:33.428275 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 15585: Not authorized: Client connection negotiation failed: client connection to 127.5.61.1:46095: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.61.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 20:18:33.428530 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 15585
I20260809 20:18:33.451319 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 15753
I20260809 20:18:33.463461 16091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:33.452449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58424 (local address 127.5.61.2:39029)
0809 20:18:33.453148 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:33.453155 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:33.453183 (+ 28us) server_negotiation.cc:407] Connection header received
0809 20:18:33.453242 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:33.453248 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:33.453449 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:33.453609 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:33.454761 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:33.455940 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:33.457117 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:33.457644 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:33.457763 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:33.460246 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:33.460254 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:33.460267 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:33.460304 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:33.462924 (+ 2620us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.61.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 20:18:33.463177 (+ 253us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58424: Request ticket server kudu/127.5.61.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":414,"threads_started":1}
W20260809 20:18:33.463593 16091 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58424: Request ticket server kudu/127.5.61.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 20:18:33.463605 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 15753: Not authorized: Client connection negotiation failed: client connection to 127.5.61.2:39029: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.61.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 20:18:33.463764 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 15753
I20260809 20:18:33.484741 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 15896
I20260809 20:18:33.497373 16092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:33.485804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44224 (local address 127.5.61.3:38439)
0809 20:18:33.486398 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:33.486405 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:33.486429 (+ 24us) server_negotiation.cc:407] Connection header received
0809 20:18:33.486490 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:33.486496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:33.486705 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:33.486859 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:33.488361 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:33.489437 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:33.490893 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:33.491428 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:33.491523 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:33.494207 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:33.494215 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:33.494228 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:33.494267 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:33.496844 (+ 2577us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.61.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 20:18:33.497093 (+ 249us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44224: Request ticket server kudu/127.5.61.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":331,"threads_started":1}
W20260809 20:18:33.497481 16092 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44224: Request ticket server kudu/127.5.61.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 20:18:33.497506 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 15896: Not authorized: Client connection negotiation failed: client connection to 127.5.61.3:38439: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.61.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 20:18:33.497674 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 15896
I20260809 20:18:33.519286 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 15504
I20260809 20:18:33.520599 15575 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:33.639673 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 15504
2026-08-09T20:18:33Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21284 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T20:18:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:18:33Z Disabled control of system clock
I20260809 20:18:33.726680 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:45637
--webserver_interface=127.5.61.62
--webserver_port=0
--builtin_ntp_servers=127.5.61.20:44365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:45637
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 20:18:34.061233 16097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:34.061504 16097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:34.061579 16097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:34.068404 16097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:18:34.068495 16097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:34.068535 16097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:34.068589 16097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:18:34.068626 16097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:18:34.077387 16097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:44365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:45637
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:45637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:34.078590 16097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:34.080293 16097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:34.091864 16103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:34.091833 16105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:34.091905 16102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:34.092753 16097 server_base.cc:1034] running on GCE node
I20260809 20:18:34.093387 16097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:34.094631 16097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:34.095834 16097 hybrid_clock.cc:648] HybridClock initialized: now 1786306714095799 us; error 34 us; skew 500 ppm
I20260809 20:18:34.096112 16097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:18:34.098192 16097 webserver.cc:460] Webserver started at http://127.5.61.62:42001/ using document root <none> and password file <none>
I20260809 20:18:34.098836 16097 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:34.098919 16097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:34.099169 16097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:18:34.101888 16097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "1960051d076145039d7367e00d3267f4"
format_stamp: "Formatted at 2026-08-09 20:18:34 on dist-test-slave-6k22"
server_key: "a37d625581335f85db39d4ce1f3fcb67"
server_key_iv: "0098a3d9ac01b82d42ab1d608286ecd7"
server_key_version: "encryptionkey@0"
I20260809 20:18:34.102644 16097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "1960051d076145039d7367e00d3267f4"
format_stamp: "Formatted at 2026-08-09 20:18:34 on dist-test-slave-6k22"
server_key: "a37d625581335f85db39d4ce1f3fcb67"
server_key_iv: "0098a3d9ac01b82d42ab1d608286ecd7"
server_key_version: "encryptionkey@0"
I20260809 20:18:34.107494 16097 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 20:18:34.110826 16111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:34.112382 16097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:18:34.112516 16097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "1960051d076145039d7367e00d3267f4"
format_stamp: "Formatted at 2026-08-09 20:18:34 on dist-test-slave-6k22"
server_key: "a37d625581335f85db39d4ce1f3fcb67"
server_key_iv: "0098a3d9ac01b82d42ab1d608286ecd7"
server_key_version: "encryptionkey@0"
I20260809 20:18:34.112685 16097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:34.126430 16097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:34.127149 16097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:34.127382 16097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:34.145383 16097 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:45637
I20260809 20:18:34.145406 16169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:45637 every 8 connection(s)
I20260809 20:18:34.146618 16097 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:18:34.148365 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 16097
I20260809 20:18:34.148581 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:18:34.149469 5364 external_mini_cluster.cc:1428] Setting key 8957487fab1975aff113fee43515e14d
I20260809 20:18:34.151189 16170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:34.163148 16173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:34.151597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45742 (local address 127.5.61.62:45637)
0809 20:18:34.152496 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:34.152506 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:34.152549 (+ 43us) server_negotiation.cc:407] Connection header received
0809 20:18:34.154080 (+ 1531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:34.154099 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:34.154511 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:34.154825 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:34.155484 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:34.156923 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:34.158852 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:34.159450 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:34.159636 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:34.160881 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:34.160888 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:34.160903 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:18:34.161477 (+ 574us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:34.161545 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:18:34.161568 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:34.161653 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:34.161916 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 20:18:34.162254 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":323,"threads_started":1}
I20260809 20:18:34.164397 16170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1960051d076145039d7367e00d3267f4: Bootstrap starting.
I20260809 20:18:34.167220 16170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1960051d076145039d7367e00d3267f4: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:34.168228 16170 log.cc:824] T 00000000000000000000000000000000 P 1960051d076145039d7367e00d3267f4: Log is configured to *not* fsync() on all Append() calls
I20260809 20:18:34.170756 16170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1960051d076145039d7367e00d3267f4: No bootstrap required, opened a new log
I20260809 20:18:34.180826 16170 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1960051d076145039d7367e00d3267f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1960051d076145039d7367e00d3267f4" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 45637 } }
I20260809 20:18:34.181269 16170 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1960051d076145039d7367e00d3267f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:34.181386 16170 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1960051d076145039d7367e00d3267f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1960051d076145039d7367e00d3267f4, State: Initialized, Role: FOLLOWER
I20260809 20:18:34.182056 16170 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1960051d076145039d7367e00d3267f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1960051d076145039d7367e00d3267f4" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 45637 } }
I20260809 20:18:34.182389 16170 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1960051d076145039d7367e00d3267f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:18:34.182524 16170 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1960051d076145039d7367e00d3267f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:18:34.182652 16170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1960051d076145039d7367e00d3267f4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:34.185075 16170 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1960051d076145039d7367e00d3267f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1960051d076145039d7367e00d3267f4" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 45637 } }
I20260809 20:18:34.185482 16170 leader_election.cc:302] T 00000000000000000000000000000000 P 1960051d076145039d7367e00d3267f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1960051d076145039d7367e00d3267f4; no voters:
I20260809 20:18:34.186029 16170 leader_election.cc:288] T 00000000000000000000000000000000 P 1960051d076145039d7367e00d3267f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:18:34.186209 16175 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1960051d076145039d7367e00d3267f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:18:34.186937 16175 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1960051d076145039d7367e00d3267f4 [term 1 LEADER]: Becoming Leader. State: Replica: 1960051d076145039d7367e00d3267f4, State: Running, Role: LEADER
I20260809 20:18:34.187479 16175 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1960051d076145039d7367e00d3267f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1960051d076145039d7367e00d3267f4" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 45637 } }
I20260809 20:18:34.189110 16170 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1960051d076145039d7367e00d3267f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:18:34.190248 16177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1960051d076145039d7367e00d3267f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1960051d076145039d7367e00d3267f4. Latest consensus state: current_term: 1 leader_uuid: "1960051d076145039d7367e00d3267f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1960051d076145039d7367e00d3267f4" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 45637 } } }
I20260809 20:18:34.190222 16176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1960051d076145039d7367e00d3267f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1960051d076145039d7367e00d3267f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1960051d076145039d7367e00d3267f4" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 45637 } } }
I20260809 20:18:34.190702 16176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1960051d076145039d7367e00d3267f4 [sys.catalog]: This master's current role is: LEADER
I20260809 20:18:34.192812 16177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1960051d076145039d7367e00d3267f4 [sys.catalog]: This master's current role is: LEADER
I20260809 20:18:34.192986 16180 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:18:34.197631 16180 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:18:34.206821 16180 catalog_manager.cc:1347] Generated new cluster ID: aa7fa1d7a8d54929a48470648fc07c9a
I20260809 20:18:34.206969 16180 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:18:34.224771 16180 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:18:34.226419 16180 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:18:34.235694 16180 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1960051d076145039d7367e00d3267f4: Generated new TSK 0
I20260809 20:18:34.236608 16180 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:18:34.246553 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:45637
--builtin_ntp_servers=127.5.61.20:44365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 20:18:34.577777 16198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:34.578049 16198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:34.578136 16198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:34.585325 16198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:34.585421 16198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:34.585518 16198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:18:34.594861 16198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:44365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:45637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:34.596131 16198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:34.597855 16198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:34.611089 16204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:34.611162 16207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:34.611194 16205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:34.612332 16198 server_base.cc:1034] running on GCE node
I20260809 20:18:34.612908 16198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:34.613816 16198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:34.615034 16198 hybrid_clock.cc:648] HybridClock initialized: now 1786306714614997 us; error 59 us; skew 500 ppm
I20260809 20:18:34.615361 16198 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:18:34.617621 16198 webserver.cc:460] Webserver started at http://127.5.61.1:42663/ using document root <none> and password file <none>
I20260809 20:18:34.618307 16198 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:34.618458 16198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:34.618790 16198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:18:34.621659 16198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "436dc376420d4ced87399276e1aa8578"
format_stamp: "Formatted at 2026-08-09 20:18:34 on dist-test-slave-6k22"
server_key: "85c72970699be49c93c14089acbb862c"
server_key_iv: "b7195ce65e24ff54944520e0d4a6906a"
server_key_version: "encryptionkey@0"
I20260809 20:18:34.622478 16198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "436dc376420d4ced87399276e1aa8578"
format_stamp: "Formatted at 2026-08-09 20:18:34 on dist-test-slave-6k22"
server_key: "85c72970699be49c93c14089acbb862c"
server_key_iv: "b7195ce65e24ff54944520e0d4a6906a"
server_key_version: "encryptionkey@0"
I20260809 20:18:34.627439 16198 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 20:18:34.630756 16216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:34.632035 16198 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:18:34.632200 16198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "436dc376420d4ced87399276e1aa8578"
format_stamp: "Formatted at 2026-08-09 20:18:34 on dist-test-slave-6k22"
server_key: "85c72970699be49c93c14089acbb862c"
server_key_iv: "b7195ce65e24ff54944520e0d4a6906a"
server_key_version: "encryptionkey@0"
I20260809 20:18:34.632407 16198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:34.654825 16198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:34.655752 16198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:34.656054 16198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:34.657043 16198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:18:34.658558 16198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:18:34.658689 16198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:34.658816 16198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:18:34.658919 16198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:34.694326 16198 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:46293
I20260809 20:18:34.694388 16337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:46293 every 8 connection(s)
I20260809 20:18:34.695741 16198 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:18:34.703034 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 16198
I20260809 20:18:34.703259 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:18:34.704126 5364 external_mini_cluster.cc:1428] Setting key afed035a43b1ceb6b9eb6aa38691ac06
I20260809 20:18:34.706743 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:45637
--builtin_ntp_servers=127.5.61.20:44365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 20:18:34.708981 16340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:34.698404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:56957 (local address 127.5.61.62:45637)
0809 20:18:34.698830 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:34.698835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:34.700161 (+ 1326us) server_negotiation.cc:407] Connection header received
0809 20:18:34.702000 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:34.702007 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:34.702099 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:34.702222 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:34.703306 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:34.704471 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:34.706090 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:34.706488 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:34.706617 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:34.708129 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:34.708134 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:34.708137 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:18:34.708155 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:34.708198 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:18:34.708210 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:34.708312 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:34.708521 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 20:18:34.708638 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":208,"threads_started":1}
I20260809 20:18:34.710075 16341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:34.699069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:45637 (local address 127.5.61.1:56957)
0809 20:18:34.699996 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:34.700052 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:34.701709 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:34.702453 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:34.702461 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:34.702545 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:34.703096 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:34.703112 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:34.704680 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:34.704688 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:34.705936 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:34.705948 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:34.706719 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:34.706726 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:34.707057 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:34.707813 (+ 756us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 20:18:34.707815 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:34.707853 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 20:18:34.707932 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:34.708311 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:34.708324 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:34.708334 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 20:18:34.708511 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 20:18:34.708944 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":262,"threads_started":1}
I20260809 20:18:34.711747 16338 heartbeater.cc:347] Connected to a master server at 127.5.61.62:45637
I20260809 20:18:34.712042 16338 heartbeater.cc:464] Registering TS with master...
I20260809 20:18:34.712845 16338 heartbeater.cc:511] Master 127.5.61.62:45637 requested a full tablet report, sending...
I20260809 20:18:34.715126 16132 ts_manager.cc:194] Registered new tserver with Master: 436dc376420d4ced87399276e1aa8578 (127.5.61.1:46293)
I20260809 20:18:34.717262 16132 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.61.1:56957
W20260809 20:18:35.069195 16342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:35.069530 16342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:35.069720 16342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:35.077049 16342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:35.077191 16342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:35.077374 16342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:18:35.086309 16342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:44365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:45637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:35.087661 16342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:35.089553 16342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:35.104045 16348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:35.104118 16353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:35.104116 16349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:35.104372 16342 server_base.cc:1034] running on GCE node
I20260809 20:18:35.105177 16342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:35.106065 16342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:35.107314 16342 hybrid_clock.cc:648] HybridClock initialized: now 1786306715107264 us; error 63 us; skew 500 ppm
I20260809 20:18:35.107699 16342 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:18:35.109951 16342 webserver.cc:460] Webserver started at http://127.5.61.2:35607/ using document root <none> and password file <none>
I20260809 20:18:35.110648 16342 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:35.110781 16342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:35.111102 16342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:18:35.114027 16342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "d278a68f6d1c46a38e440565ea39793a"
format_stamp: "Formatted at 2026-08-09 20:18:35 on dist-test-slave-6k22"
server_key: "d22baaade364d1710cb6552a99f5a2f1"
server_key_iv: "d19883a0c51fa44fc2341bedc47b899f"
server_key_version: "encryptionkey@0"
I20260809 20:18:35.114847 16342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "d278a68f6d1c46a38e440565ea39793a"
format_stamp: "Formatted at 2026-08-09 20:18:35 on dist-test-slave-6k22"
server_key: "d22baaade364d1710cb6552a99f5a2f1"
server_key_iv: "d19883a0c51fa44fc2341bedc47b899f"
server_key_version: "encryptionkey@0"
I20260809 20:18:35.120079 16342 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260809 20:18:35.123359 16360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:35.124686 16342 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:18:35.124876 16342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "d278a68f6d1c46a38e440565ea39793a"
format_stamp: "Formatted at 2026-08-09 20:18:35 on dist-test-slave-6k22"
server_key: "d22baaade364d1710cb6552a99f5a2f1"
server_key_iv: "d19883a0c51fa44fc2341bedc47b899f"
server_key_version: "encryptionkey@0"
I20260809 20:18:35.125088 16342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:35.154306 16342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:35.155311 16342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:35.155630 16342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:35.156702 16342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:18:35.158524 16342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:18:35.158630 16342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:35.158728 16342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:18:35.158810 16342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:35.193312 16342 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:40453
I20260809 20:18:35.193421 16480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:40453 every 8 connection(s)
I20260809 20:18:35.194571 16342 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:18:35.194880 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 16342
I20260809 20:18:35.195051 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:18:35.195708 5364 external_mini_cluster.cc:1428] Setting key f8018087c94efb5b269c7f00b3df88db
I20260809 20:18:35.198897 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:45637
--builtin_ntp_servers=127.5.61.20:44365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 20:18:35.207819 16340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:35.197598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:37343 (local address 127.5.61.62:45637)
0809 20:18:35.197819 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:35.197825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:35.199473 (+ 1648us) server_negotiation.cc:407] Connection header received
0809 20:18:35.201395 (+ 1922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:35.201401 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:35.201499 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:35.201599 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:35.202637 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:35.203345 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:35.204890 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:35.205344 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:35.205484 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:35.207035 (+ 1551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:35.207040 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:35.207043 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:18:35.207060 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:35.207091 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:18:35.207102 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:35.207171 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:35.207444 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 20:18:35.207519 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 20:18:35.209196 16483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:35.198242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:45637 (local address 127.5.61.2:37343)
0809 20:18:35.199329 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:35.199365 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:35.201148 (+ 1783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:35.201834 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:35.201846 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:35.201947 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:35.202474 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:35.202489 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:35.203506 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:35.203511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:35.204744 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:35.204756 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:35.205607 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:35.205617 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:35.205962 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:35.206693 (+ 731us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 20:18:35.206698 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:35.206749 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 20:18:35.206846 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:35.207200 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:35.207212 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:35.207222 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 20:18:35.207438 (+ 216us) client_negotiation.cc:240] Negotiation successful
0809 20:18:35.207849 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":340,"threads_started":1}
I20260809 20:18:35.210575 16481 heartbeater.cc:347] Connected to a master server at 127.5.61.62:45637
I20260809 20:18:35.210883 16481 heartbeater.cc:464] Registering TS with master...
I20260809 20:18:35.211647 16481 heartbeater.cc:511] Master 127.5.61.62:45637 requested a full tablet report, sending...
I20260809 20:18:35.213003 16132 ts_manager.cc:194] Registered new tserver with Master: d278a68f6d1c46a38e440565ea39793a (127.5.61.2:40453)
I20260809 20:18:35.213934 16132 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.61.2:37343
W20260809 20:18:35.533113 16484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:35.533376 16484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:35.533524 16484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:35.540760 16484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:35.540848 16484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:35.541034 16484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:18:35.549655 16484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:44365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:45637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:35.550912 16484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:35.552708 16484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:35.565953 16490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:35.566007 16491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:35.565913 16493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:35.566880 16484 server_base.cc:1034] running on GCE node
I20260809 20:18:35.567467 16484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:35.568406 16484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:35.569638 16484 hybrid_clock.cc:648] HybridClock initialized: now 1786306715569568 us; error 53 us; skew 500 ppm
I20260809 20:18:35.569991 16484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:18:35.574127 16484 webserver.cc:460] Webserver started at http://127.5.61.3:42987/ using document root <none> and password file <none>
I20260809 20:18:35.574867 16484 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:35.575003 16484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:35.575368 16484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:18:35.578341 16484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "1bc0c4fc87c84b0ea120f9ae3c1654a2"
format_stamp: "Formatted at 2026-08-09 20:18:35 on dist-test-slave-6k22"
server_key: "5b1fa2e3c7cba084578b1f8a9d1e1721"
server_key_iv: "70438ee820de42d66438cf2ea48d19f8"
server_key_version: "encryptionkey@0"
I20260809 20:18:35.579203 16484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "1bc0c4fc87c84b0ea120f9ae3c1654a2"
format_stamp: "Formatted at 2026-08-09 20:18:35 on dist-test-slave-6k22"
server_key: "5b1fa2e3c7cba084578b1f8a9d1e1721"
server_key_iv: "70438ee820de42d66438cf2ea48d19f8"
server_key_version: "encryptionkey@0"
I20260809 20:18:35.584614 16484 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 20:18:35.588124 16501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:35.589763 16484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 20:18:35.589943 16484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "1bc0c4fc87c84b0ea120f9ae3c1654a2"
format_stamp: "Formatted at 2026-08-09 20:18:35 on dist-test-slave-6k22"
server_key: "5b1fa2e3c7cba084578b1f8a9d1e1721"
server_key_iv: "70438ee820de42d66438cf2ea48d19f8"
server_key_version: "encryptionkey@0"
I20260809 20:18:35.590169 16484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:35.602412 16484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:35.603248 16484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:35.603518 16484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:35.604482 16484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:18:35.605897 16484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:18:35.606026 16484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:35.606151 16484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:18:35.606248 16484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:35.639778 16484 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:35291
I20260809 20:18:35.639837 16624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:35291 every 8 connection(s)
I20260809 20:18:35.641062 16484 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:18:35.646844 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 16484
I20260809 20:18:35.647042 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:18:35.647720 5364 external_mini_cluster.cc:1428] Setting key 713588c9ede18aae7da135a0b7343d0b
I20260809 20:18:35.653980 16340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:35.643522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:35427 (local address 127.5.61.62:45637)
0809 20:18:35.643736 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:35.643742 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:35.645098 (+ 1356us) server_negotiation.cc:407] Connection header received
0809 20:18:35.647025 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:35.647034 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:35.647138 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:35.647256 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:35.648633 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:35.649629 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:35.651498 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:35.651996 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:35.652094 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:35.653330 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:35.653335 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:35.653338 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:18:35.653356 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:35.653388 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:18:35.653399 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:35.653470 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:35.653660 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 20:18:35.653750 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 20:18:35.654887 16628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:35.644084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:45637 (local address 127.5.61.3:35427)
0809 20:18:35.644948 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:35.644985 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:35.646708 (+ 1723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:35.647522 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:35.647533 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:35.647674 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:35.648424 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:35.648445 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:35.649807 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:35.649812 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:35.651354 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:35.651374 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:35.652188 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:35.652193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:35.652393 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:35.653044 (+ 651us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 20:18:35.653047 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:35.653089 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 20:18:35.653160 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:35.653485 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:35.653493 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:35.653498 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 20:18:35.653639 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 20:18:35.653904 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":262,"threads_started":1}
I20260809 20:18:35.656299 16625 heartbeater.cc:347] Connected to a master server at 127.5.61.62:45637
I20260809 20:18:35.656579 16625 heartbeater.cc:464] Registering TS with master...
I20260809 20:18:35.657248 16625 heartbeater.cc:511] Master 127.5.61.62:45637 requested a full tablet report, sending...
I20260809 20:18:35.658561 16132 ts_manager.cc:194] Registered new tserver with Master: 1bc0c4fc87c84b0ea120f9ae3c1654a2 (127.5.61.3:35291)
I20260809 20:18:35.659333 16132 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.61.3:35427
I20260809 20:18:35.662952 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 20:18:35.668992 16340 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:35.667396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45744 (local address 127.5.61.62:45637)
0809 20:18:35.667617 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:35.667625 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:35.668096 (+ 471us) server_negotiation.cc:407] Connection header received
0809 20:18:35.668213 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:35.668216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:35.668291 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:35.668367 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:35.668763 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45744: BlockingRecv error: recv got EOF from 127.0.0.1:45744 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 20:18:35.670028 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 16198
I20260809 20:18:35.677488 16637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:35.670905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58738 (local address 127.5.61.1:46293)
0809 20:18:35.671455 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:35.671461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:35.671485 (+ 24us) server_negotiation.cc:407] Connection header received
0809 20:18:35.671539 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:35.671544 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:35.671854 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:35.672000 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:35.672491 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:35.673512 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:35.674666 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:35.675178 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:35.675272 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:35.676268 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:35.676274 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:35.676284 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:18:35.676840 (+ 556us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:35.676894 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:18:35.676912 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:35.677000 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:35.677114 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 20:18:35.677214 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":293,"threads_started":1}
I20260809 20:18:35.678637 16333 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:35.760476 16338 heartbeater.cc:503] Master 127.5.61.62:45637 was elected leader, sending a full tablet report...
I20260809 20:18:35.761616 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 16198
W20260809 20:18:35.780378 16112 connection.cc:535] server connection from 127.5.61.1:56957 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 20:18:35.780753 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 16342
I20260809 20:18:35.788563 16639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:35.781666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58396 (local address 127.5.61.2:40453)
0809 20:18:35.782339 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:35.782345 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:35.782372 (+ 27us) server_negotiation.cc:407] Connection header received
0809 20:18:35.782435 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:35.782441 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:35.782668 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:35.782802 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:35.783130 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:35.784251 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:35.785516 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:35.786059 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:35.786157 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:35.787120 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:35.787128 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:35.787140 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:18:35.787876 (+ 736us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:35.787920 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:18:35.787939 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:35.788041 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:35.788161 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 20:18:35.788275 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":391,"threads_started":1}
I20260809 20:18:35.789891 16476 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:35.872912 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 16342
I20260809 20:18:35.894335 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 16484
I20260809 20:18:35.902308 16641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:35.895376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38402 (local address 127.5.61.3:35291)
0809 20:18:35.895991 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:35.895998 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:35.896027 (+ 29us) server_negotiation.cc:407] Connection header received
0809 20:18:35.896086 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:35.896092 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:35.896301 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:35.896433 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:35.896822 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:35.897895 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:35.899128 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:35.899765 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:35.899871 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:35.900921 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:35.900929 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:35.900941 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:18:35.901583 (+ 642us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:35.901628 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:18:35.901672 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:35.901800 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:35.901885 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 20:18:35.901999 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":328,"threads_started":1}
I20260809 20:18:35.903469 16620 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:35.984198 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 16484
I20260809 20:18:36.005187 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 16097
I20260809 20:18:36.006458 16165 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:36.108198 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 16097
2026-08-09T20:18:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2461 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T20:18:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:18:36Z Disabled control of system clock
I20260809 20:18:36.189149 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:37715
--webserver_interface=127.5.61.62
--webserver_port=0
--builtin_ntp_servers=127.5.61.20:38629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:37715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 20:18:36.513069 16648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:36.513374 16648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:36.513465 16648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:36.520653 16648 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:18:36.520756 16648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:36.520793 16648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:36.520846 16648 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:18:36.520889 16648 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:18:36.529781 16648 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:38629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:37715
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:37715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:36.530965 16648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:36.532739 16648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:36.544497 16654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:36.544467 16655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:36.544516 16658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:36.545617 16648 server_base.cc:1034] running on GCE node
I20260809 20:18:36.546271 16648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:36.547534 16648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:36.548799 16648 hybrid_clock.cc:648] HybridClock initialized: now 1786306716548750 us; error 50 us; skew 500 ppm
I20260809 20:18:36.549144 16648 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:18:36.551255 16648 webserver.cc:460] Webserver started at http://127.5.61.62:41051/ using document root <none> and password file <none>
I20260809 20:18:36.552007 16648 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:36.552129 16648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:36.552470 16648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:18:36.555262 16648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "8cadeb51ddb14cb4a2e7ba0f4930448b"
format_stamp: "Formatted at 2026-08-09 20:18:36 on dist-test-slave-6k22"
server_key: "110b2828aa0ea16526f656acd7a741c3"
server_key_iv: "95a029ee3d09f0690ae91fadd596287d"
server_key_version: "encryptionkey@0"
I20260809 20:18:36.556109 16648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "8cadeb51ddb14cb4a2e7ba0f4930448b"
format_stamp: "Formatted at 2026-08-09 20:18:36 on dist-test-slave-6k22"
server_key: "110b2828aa0ea16526f656acd7a741c3"
server_key_iv: "95a029ee3d09f0690ae91fadd596287d"
server_key_version: "encryptionkey@0"
I20260809 20:18:36.561040 16648 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 20:18:36.564293 16665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:36.565557 16648 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:18:36.565729 16648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "8cadeb51ddb14cb4a2e7ba0f4930448b"
format_stamp: "Formatted at 2026-08-09 20:18:36 on dist-test-slave-6k22"
server_key: "110b2828aa0ea16526f656acd7a741c3"
server_key_iv: "95a029ee3d09f0690ae91fadd596287d"
server_key_version: "encryptionkey@0"
I20260809 20:18:36.565949 16648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:36.571755 16648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:36.572512 16648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:36.572813 16648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:36.590998 16648 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:37715
I20260809 20:18:36.591030 16720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:37715 every 8 connection(s)
I20260809 20:18:36.592456 16648 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:18:36.595969 16721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:36.599881 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 16648
I20260809 20:18:36.600099 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:18:36.600713 5364 external_mini_cluster.cc:1428] Setting key 3b21020280248b4f0cdc7c86fd8d6be9
I20260809 20:18:36.606630 16721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cadeb51ddb14cb4a2e7ba0f4930448b: Bootstrap starting.
I20260809 20:18:36.608664 16724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:36.602623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42148 (local address 127.5.61.62:37715)
0809 20:18:36.603635 (+ 1012us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:36.603644 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:36.603688 (+ 44us) server_negotiation.cc:407] Connection header received
0809 20:18:36.605334 (+ 1646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:36.605355 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:36.605726 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:36.606024 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:36.607157 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:36.607163 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:36.607174 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:18:36.607188 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:36.607243 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:18:36.607258 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:36.607430 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:36.607529 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 20:18:36.607844 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":360,"threads_started":1}
I20260809 20:18:36.609886 16721 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cadeb51ddb14cb4a2e7ba0f4930448b: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:36.610996 16721 log.cc:824] T 00000000000000000000000000000000 P 8cadeb51ddb14cb4a2e7ba0f4930448b: Log is configured to *not* fsync() on all Append() calls
I20260809 20:18:36.613526 16721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cadeb51ddb14cb4a2e7ba0f4930448b: No bootstrap required, opened a new log
I20260809 20:18:36.620761 16721 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cadeb51ddb14cb4a2e7ba0f4930448b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cadeb51ddb14cb4a2e7ba0f4930448b" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37715 } }
I20260809 20:18:36.621105 16721 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cadeb51ddb14cb4a2e7ba0f4930448b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:36.621208 16721 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cadeb51ddb14cb4a2e7ba0f4930448b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cadeb51ddb14cb4a2e7ba0f4930448b, State: Initialized, Role: FOLLOWER
I20260809 20:18:36.621752 16721 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cadeb51ddb14cb4a2e7ba0f4930448b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cadeb51ddb14cb4a2e7ba0f4930448b" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37715 } }
I20260809 20:18:36.622005 16721 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cadeb51ddb14cb4a2e7ba0f4930448b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:18:36.622146 16721 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cadeb51ddb14cb4a2e7ba0f4930448b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:18:36.622311 16721 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cadeb51ddb14cb4a2e7ba0f4930448b [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:36.624491 16721 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cadeb51ddb14cb4a2e7ba0f4930448b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cadeb51ddb14cb4a2e7ba0f4930448b" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37715 } }
I20260809 20:18:36.624958 16721 leader_election.cc:302] T 00000000000000000000000000000000 P 8cadeb51ddb14cb4a2e7ba0f4930448b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cadeb51ddb14cb4a2e7ba0f4930448b; no voters:
I20260809 20:18:36.625536 16721 leader_election.cc:288] T 00000000000000000000000000000000 P 8cadeb51ddb14cb4a2e7ba0f4930448b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:18:36.625635 16726 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cadeb51ddb14cb4a2e7ba0f4930448b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:18:36.626389 16726 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cadeb51ddb14cb4a2e7ba0f4930448b [term 1 LEADER]: Becoming Leader. State: Replica: 8cadeb51ddb14cb4a2e7ba0f4930448b, State: Running, Role: LEADER
I20260809 20:18:36.626996 16726 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cadeb51ddb14cb4a2e7ba0f4930448b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cadeb51ddb14cb4a2e7ba0f4930448b" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37715 } }
I20260809 20:18:36.627578 16721 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cadeb51ddb14cb4a2e7ba0f4930448b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:18:36.629989 16728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cadeb51ddb14cb4a2e7ba0f4930448b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cadeb51ddb14cb4a2e7ba0f4930448b. Latest consensus state: current_term: 1 leader_uuid: "8cadeb51ddb14cb4a2e7ba0f4930448b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cadeb51ddb14cb4a2e7ba0f4930448b" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37715 } } }
I20260809 20:18:36.630775 16728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cadeb51ddb14cb4a2e7ba0f4930448b [sys.catalog]: This master's current role is: LEADER
I20260809 20:18:36.630704 16727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cadeb51ddb14cb4a2e7ba0f4930448b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cadeb51ddb14cb4a2e7ba0f4930448b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cadeb51ddb14cb4a2e7ba0f4930448b" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37715 } } }
I20260809 20:18:36.631590 16727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cadeb51ddb14cb4a2e7ba0f4930448b [sys.catalog]: This master's current role is: LEADER
I20260809 20:18:36.632443 16735 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:18:36.637467 16735 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:18:36.645788 16735 catalog_manager.cc:1347] Generated new cluster ID: 3a9f769262784795bfd697caee81c9ec
I20260809 20:18:36.645888 16735 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:18:36.668896 16735 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:18:36.669404 16735 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:18:36.683636 16735 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cadeb51ddb14cb4a2e7ba0f4930448b: Generated new TSK 0
I20260809 20:18:36.684408 16735 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:18:36.697189 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:37715
--builtin_ntp_servers=127.5.61.20:38629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 20:18:37.051671 16745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:37.051956 16745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:37.052156 16745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:37.059928 16745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:37.060034 16745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:37.060216 16745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:18:37.069234 16745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:38629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:37715
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:37.070601 16745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:37.072400 16745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:37.085963 16750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:37.085919 16754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:37.086009 16752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:37.088902 16745 server_base.cc:1034] running on GCE node
I20260809 20:18:37.089486 16745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:37.090471 16745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:37.091679 16745 hybrid_clock.cc:648] HybridClock initialized: now 1786306717091639 us; error 49 us; skew 500 ppm
I20260809 20:18:37.092039 16745 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:18:37.094252 16745 webserver.cc:460] Webserver started at http://127.5.61.1:37321/ using document root <none> and password file <none>
I20260809 20:18:37.094972 16745 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:37.095105 16745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:37.095434 16745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:18:37.100521 16745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "8107cf6a96a4484fb1088ce6b9756991"
format_stamp: "Formatted at 2026-08-09 20:18:37 on dist-test-slave-6k22"
server_key: "5441c9b5670c7d98bf01977a1d650a70"
server_key_iv: "7f36699323a6069addfcab0ab37e8af4"
server_key_version: "encryptionkey@0"
I20260809 20:18:37.101303 16745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "8107cf6a96a4484fb1088ce6b9756991"
format_stamp: "Formatted at 2026-08-09 20:18:37 on dist-test-slave-6k22"
server_key: "5441c9b5670c7d98bf01977a1d650a70"
server_key_iv: "7f36699323a6069addfcab0ab37e8af4"
server_key_version: "encryptionkey@0"
I20260809 20:18:37.106178 16745 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 20:18:37.109761 16760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:37.111058 16745 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 20:18:37.111238 16745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "8107cf6a96a4484fb1088ce6b9756991"
format_stamp: "Formatted at 2026-08-09 20:18:37 on dist-test-slave-6k22"
server_key: "5441c9b5670c7d98bf01977a1d650a70"
server_key_iv: "7f36699323a6069addfcab0ab37e8af4"
server_key_version: "encryptionkey@0"
I20260809 20:18:37.111460 16745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:37.117086 16745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:37.117911 16745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:37.118176 16745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:37.119155 16745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:18:37.120640 16745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:18:37.120779 16745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:37.120903 16745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:18:37.120992 16745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:37.155745 16745 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:41505
I20260809 20:18:37.155784 16888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:41505 every 8 connection(s)
I20260809 20:18:37.157089 16745 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:18:37.159405 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 16745
I20260809 20:18:37.159695 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:18:37.160537 5364 external_mini_cluster.cc:1428] Setting key 7e6be39f4d2657b2952bbd50374f205a
I20260809 20:18:37.163158 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:37715
--builtin_ntp_servers=127.5.61.20:38629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 20:18:37.165629 16891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:37.159676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:45137 (local address 127.5.61.62:37715)
0809 20:18:37.160208 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:37.160215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:37.161607 (+ 1392us) server_negotiation.cc:407] Connection header received
0809 20:18:37.163276 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:37.163282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:37.163345 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:37.163465 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:37.164978 (+ 1513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:37.164983 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:37.164986 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:18:37.165001 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:37.165043 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:18:37.165055 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:37.165151 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:37.165374 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 20:18:37.165422 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":253,"threads_started":1}
I20260809 20:18:37.166800 16892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:37.160336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:37715 (local address 127.5.61.1:45137)
0809 20:18:37.161444 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:37.161489 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:37.163024 (+ 1535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:37.163758 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:37.163773 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:37.163889 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:37.164624 (+ 735us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 20:18:37.164630 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:37.164682 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 20:18:37.164805 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:37.165154 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:37.165166 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:37.165175 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 20:18:37.165439 (+ 264us) client_negotiation.cc:240] Negotiation successful
0809 20:18:37.165679 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":357,"threads_started":1}
I20260809 20:18:37.168216 16889 heartbeater.cc:347] Connected to a master server at 127.5.61.62:37715
I20260809 20:18:37.168503 16889 heartbeater.cc:464] Registering TS with master...
I20260809 20:18:37.169214 16889 heartbeater.cc:511] Master 127.5.61.62:37715 requested a full tablet report, sending...
I20260809 20:18:37.171448 16684 ts_manager.cc:194] Registered new tserver with Master: 8107cf6a96a4484fb1088ce6b9756991 (127.5.61.1:41505)
W20260809 20:18:37.499739 16893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:37.500010 16893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:37.500099 16893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:37.507131 16893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:37.507223 16893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:37.507335 16893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:18:37.516319 16893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:38629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:37715
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:37.517521 16893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:37.519337 16893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:37.533022 16898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:37.533107 16899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:37.533517 16901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:37.534145 16893 server_base.cc:1034] running on GCE node
I20260809 20:18:37.534730 16893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:37.535813 16893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:37.537035 16893 hybrid_clock.cc:648] HybridClock initialized: now 1786306717536994 us; error 55 us; skew 500 ppm
I20260809 20:18:37.537355 16893 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:18:37.539618 16893 webserver.cc:460] Webserver started at http://127.5.61.2:46867/ using document root <none> and password file <none>
I20260809 20:18:37.540340 16893 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:37.540474 16893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:37.540823 16893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:18:37.543690 16893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "4423a287958746d386b508a45b61b0c2"
format_stamp: "Formatted at 2026-08-09 20:18:37 on dist-test-slave-6k22"
server_key: "30e089dfb566da30f2b2e82320ad1e26"
server_key_iv: "db44fe9e8e091059b85165ec9c31a2f7"
server_key_version: "encryptionkey@0"
I20260809 20:18:37.544514 16893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "4423a287958746d386b508a45b61b0c2"
format_stamp: "Formatted at 2026-08-09 20:18:37 on dist-test-slave-6k22"
server_key: "30e089dfb566da30f2b2e82320ad1e26"
server_key_iv: "db44fe9e8e091059b85165ec9c31a2f7"
server_key_version: "encryptionkey@0"
I20260809 20:18:37.549854 16893 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 20:18:37.553474 16911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:37.554764 16893 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 20:18:37.554937 16893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "4423a287958746d386b508a45b61b0c2"
format_stamp: "Formatted at 2026-08-09 20:18:37 on dist-test-slave-6k22"
server_key: "30e089dfb566da30f2b2e82320ad1e26"
server_key_iv: "db44fe9e8e091059b85165ec9c31a2f7"
server_key_version: "encryptionkey@0"
I20260809 20:18:37.555153 16893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:37.560747 16893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:37.561524 16893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:37.561806 16893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:37.562728 16893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:18:37.564172 16893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:18:37.564302 16893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:37.564440 16893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:18:37.564538 16893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:37.598148 16893 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:33995
I20260809 20:18:37.598178 17031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:33995 every 8 connection(s)
I20260809 20:18:37.599848 16893 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:18:37.601470 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 16893
I20260809 20:18:37.601687 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:18:37.602252 5364 external_mini_cluster.cc:1428] Setting key 1acaa3f59f4cf01ad898c2090a87340c
I20260809 20:18:37.605320 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:37715
--builtin_ntp_servers=127.5.61.20:38629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 20:18:37.616313 16891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:37.603328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:52863 (local address 127.5.61.62:37715)
0809 20:18:37.603514 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:37.603520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:37.605076 (+ 1556us) server_negotiation.cc:407] Connection header received
0809 20:18:37.607866 (+ 2790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:37.607875 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:37.607989 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:37.608064 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:37.615458 (+ 7394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:37.615464 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:37.615486 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:18:37.615511 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:37.615553 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:18:37.615609 (+ 56us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:37.615746 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:37.615956 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 20:18:37.616038 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 20:18:37.617408 17034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:37.603778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:37715 (local address 127.5.61.2:52863)
0809 20:18:37.604902 (+ 1124us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:37.604951 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:37.607536 (+ 2585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:37.608564 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:37.608577 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:37.608684 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:37.615085 (+ 6401us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 20:18:37.615090 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:37.615142 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 20:18:37.615263 (+ 121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:37.615717 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:37.615729 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:37.615736 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 20:18:37.616039 (+ 303us) client_negotiation.cc:240] Negotiation successful
0809 20:18:37.616304 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":367,"threads_started":1}
I20260809 20:18:37.619165 17032 heartbeater.cc:347] Connected to a master server at 127.5.61.62:37715
I20260809 20:18:37.619491 17032 heartbeater.cc:464] Registering TS with master...
I20260809 20:18:37.620106 17032 heartbeater.cc:511] Master 127.5.61.62:37715 requested a full tablet report, sending...
I20260809 20:18:37.621510 16684 ts_manager.cc:194] Registered new tserver with Master: 4423a287958746d386b508a45b61b0c2 (127.5.61.2:33995)
W20260809 20:18:37.942616 17035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:37.942914 17035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:37.943032 17035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:37.950984 17035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:37.951092 17035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:37.951237 17035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:18:37.961283 17035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:38629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:37715
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:37.962603 17035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:37.964629 17035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:37.978422 17040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:37.978345 17041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:37.978444 17043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:37.979192 17035 server_base.cc:1034] running on GCE node
I20260809 20:18:37.979907 17035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:37.980840 17035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:37.982069 17035 hybrid_clock.cc:648] HybridClock initialized: now 1786306717982034 us; error 55 us; skew 500 ppm
I20260809 20:18:37.982386 17035 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:18:37.984686 17035 webserver.cc:460] Webserver started at http://127.5.61.3:36489/ using document root <none> and password file <none>
I20260809 20:18:37.985383 17035 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:37.985536 17035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:37.985888 17035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:18:37.988762 17035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "d76a7c1bf98544d69b0a8ce060e3349e"
format_stamp: "Formatted at 2026-08-09 20:18:37 on dist-test-slave-6k22"
server_key: "4b8c600ce77c5e84dc68cfce8fc7ac93"
server_key_iv: "49bee7200ca38dae467c7ff6f5bab147"
server_key_version: "encryptionkey@0"
I20260809 20:18:37.989574 17035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "d76a7c1bf98544d69b0a8ce060e3349e"
format_stamp: "Formatted at 2026-08-09 20:18:37 on dist-test-slave-6k22"
server_key: "4b8c600ce77c5e84dc68cfce8fc7ac93"
server_key_iv: "49bee7200ca38dae467c7ff6f5bab147"
server_key_version: "encryptionkey@0"
I20260809 20:18:37.994545 17035 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 20:18:37.998044 17050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:37.999312 17035 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 20:18:37.999480 17035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "d76a7c1bf98544d69b0a8ce060e3349e"
format_stamp: "Formatted at 2026-08-09 20:18:37 on dist-test-slave-6k22"
server_key: "4b8c600ce77c5e84dc68cfce8fc7ac93"
server_key_iv: "49bee7200ca38dae467c7ff6f5bab147"
server_key_version: "encryptionkey@0"
I20260809 20:18:37.999763 17035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:38.005434 17035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:38.006193 17035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:38.006461 17035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:38.007316 17035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:18:38.008682 17035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:18:38.008821 17035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:38.008944 17035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:18:38.009032 17035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:38.042096 17035 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:40097
I20260809 20:18:38.042150 17179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:40097 every 8 connection(s)
I20260809 20:18:38.043448 17035 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:18:38.051225 16891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:38.046056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:47891 (local address 127.5.61.62:37715)
0809 20:18:38.046250 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:38.046254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:38.047554 (+ 1300us) server_negotiation.cc:407] Connection header received
0809 20:18:38.049199 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:38.049204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:38.049258 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:38.049348 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:38.050620 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:38.050624 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:38.050628 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:18:38.050644 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:38.050676 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:18:38.050688 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:38.050788 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:38.050953 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 20:18:38.051003 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 20:18:38.052187 17182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:38.046545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:37715 (local address 127.5.61.3:47891)
0809 20:18:38.047397 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:38.047443 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:38.048938 (+ 1495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:38.049543 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:38.049550 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:38.049636 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:38.050252 (+ 616us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 20:18:38.050254 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:38.050302 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 20:18:38.050381 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:38.050782 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:38.050789 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:38.050794 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 20:18:38.051000 (+ 206us) client_negotiation.cc:240] Negotiation successful
0809 20:18:38.051189 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":259,"threads_started":1}
I20260809 20:18:38.052847 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 17035
I20260809 20:18:38.053059 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:18:38.053519 5364 external_mini_cluster.cc:1428] Setting key 61a64a26cd5674aef642e5e4a5ed86b9
I20260809 20:18:38.053560 17180 heartbeater.cc:347] Connected to a master server at 127.5.61.62:37715
I20260809 20:18:38.053848 17180 heartbeater.cc:464] Registering TS with master...
I20260809 20:18:38.054559 17180 heartbeater.cc:511] Master 127.5.61.62:37715 requested a full tablet report, sending...
I20260809 20:18:38.056049 16683 ts_manager.cc:194] Registered new tserver with Master: d76a7c1bf98544d69b0a8ce060e3349e (127.5.61.3:40097)
I20260809 20:18:38.069006 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 20:18:38.075361 16891 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:18:38.073674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42156 (local address 127.5.61.62:37715)
0809 20:18:38.073887 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:38.073893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:38.074457 (+ 564us) server_negotiation.cc:407] Connection header received
0809 20:18:38.074585 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:38.074589 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:38.074644 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:38.074743 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:38.075085 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42156: BlockingRecv error: recv got EOF from 127.0.0.1:42156 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 20:18:38.075718 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 16745
I20260809 20:18:38.079687 17190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:38.076845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36560 (local address 127.5.61.1:41505)
0809 20:18:38.077386 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:38.077392 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:38.077436 (+ 44us) server_negotiation.cc:407] Connection header received
0809 20:18:38.077501 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:38.077506 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:38.077698 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:38.077873 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:38.078897 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:38.078905 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:38.078915 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:18:38.078929 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:38.078972 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:18:38.078986 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:38.079132 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:38.079238 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 20:18:38.079343 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":287,"threads_started":1}
I20260809 20:18:38.080856 16884 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:38.163478 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 16745
I20260809 20:18:38.184517 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 16893
I20260809 20:18:38.188063 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:38.185473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40900 (local address 127.5.61.2:33995)
0809 20:18:38.185959 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:38.185966 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:38.185993 (+ 27us) server_negotiation.cc:407] Connection header received
0809 20:18:38.186063 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:38.186069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:38.186307 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:38.186436 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:38.187311 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:38.187317 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:38.187328 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:18:38.187339 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:38.187380 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:18:38.187396 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:38.187505 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:38.187656 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 20:18:38.187795 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":243,"threads_started":1}
I20260809 20:18:38.189314 17027 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:38.270903 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 16893
I20260809 20:18:38.291296 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 17035
I20260809 20:18:38.294966 17194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:38.292211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36238 (local address 127.5.61.3:40097)
0809 20:18:38.292854 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:38.292882 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:38.292913 (+ 31us) server_negotiation.cc:407] Connection header received
0809 20:18:38.292980 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:38.292986 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:38.293198 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:38.293329 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:38.294341 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:38.294347 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:38.294358 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:18:38.294370 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:38.294409 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:18:38.294424 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:38.294569 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:38.294615 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 20:18:38.294710 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":305,"threads_started":1}
I20260809 20:18:38.296367 17175 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:38.377915 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 17035
I20260809 20:18:38.398434 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 16648
I20260809 20:18:38.399735 16716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:38.501969 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 16648
2026-08-09T20:18:38Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2394 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:18:38 dist-test-slave-6k22 krb5kdc[17199](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:18:38 dist-test-slave-6k22 krb5kdc[17199](info): set up 2 sockets
Aug 09 20:18:38 dist-test-slave-6k22 krb5kdc[17199](info): commencing operation
krb5kdc: starting...
W20260809 20:18:39.701040 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:18:39 dist-test-slave-6k22 krb5kdc[17199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:18:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:18:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:18:39.955703 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:34143
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:34273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:34143
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:18:40.280177 17215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:40.280511 17215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:40.280678 17215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:40.287703 17215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:18:40.287829 17215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:40.287959 17215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:40.288075 17215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:18:40.288185 17215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:18:40.297483 17215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:34273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:34143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:34143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:40.298795 17215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:40.300094 17215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:40.311766 17224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:40.311810 17220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:40.311719 17222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:40.312721 17215 server_base.cc:1034] running on GCE node
I20260809 20:18:40.313470 17215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:40.314746 17215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:40.315956 17215 hybrid_clock.cc:648] HybridClock initialized: now 1786306720315888 us; error 55 us; skew 500 ppm
I20260809 20:18:40.316322 17215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:18:40 dist-test-slave-6k22 krb5kdc[17199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306720, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:18:40.319504 17215 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:18:40.320878 17215 webserver.cc:460] Webserver started at http://127.5.61.62:42463/ using document root <none> and password file <none>
I20260809 20:18:40.321472 17215 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:40.321573 17215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:40.321810 17215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:18:40.324770 17215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "e06586bcc76f466eae83026c0c0d7491"
format_stamp: "Formatted at 2026-08-09 20:18:40 on dist-test-slave-6k22"
server_key: "c7b15cc1760bdd9b5767988f08848953"
server_key_iv: "4c1232960a3a02e722d9e6f298863ebb"
server_key_version: "encryptionkey@0"
I20260809 20:18:40.325526 17215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "e06586bcc76f466eae83026c0c0d7491"
format_stamp: "Formatted at 2026-08-09 20:18:40 on dist-test-slave-6k22"
server_key: "c7b15cc1760bdd9b5767988f08848953"
server_key_iv: "4c1232960a3a02e722d9e6f298863ebb"
server_key_version: "encryptionkey@0"
I20260809 20:18:40.330400 17215 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 20:18:40.333679 17231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:40.335000 17215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 20:18:40.335181 17215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "e06586bcc76f466eae83026c0c0d7491"
format_stamp: "Formatted at 2026-08-09 20:18:40 on dist-test-slave-6k22"
server_key: "c7b15cc1760bdd9b5767988f08848953"
server_key_iv: "4c1232960a3a02e722d9e6f298863ebb"
server_key_version: "encryptionkey@0"
I20260809 20:18:40.335381 17215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:40.370808 17215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:40.375278 17215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:40.375638 17215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:40.394323 17215 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:34143
I20260809 20:18:40.394366 17286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:34143 every 8 connection(s)
I20260809 20:18:40.395752 17215 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:18:40.399726 17287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:40.400887 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 17215
I20260809 20:18:40.401134 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:18:40.401997 5364 external_mini_cluster.cc:1428] Setting key ed9b76eb5c21f7b17d4db2a522aea379
I20260809 20:18:40.414335 17287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e06586bcc76f466eae83026c0c0d7491: Bootstrap starting.
Aug 09 20:18:40 dist-test-slave-6k22 krb5kdc[17199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:18:40.417945 17287 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e06586bcc76f466eae83026c0c0d7491: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:40.419173 17287 log.cc:824] T 00000000000000000000000000000000 P e06586bcc76f466eae83026c0c0d7491: Log is configured to *not* fsync() on all Append() calls
I20260809 20:18:40.422030 17287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e06586bcc76f466eae83026c0c0d7491: No bootstrap required, opened a new log
I20260809 20:18:40.425565 17290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:40.404232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38208 (local address 127.5.61.62:34143)
0809 20:18:40.405080 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:40.405092 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:40.405132 (+ 40us) server_negotiation.cc:407] Connection header received
0809 20:18:40.406711 (+ 1579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:40.406733 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:40.407127 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:40.407466 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:40.409276 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:40.410537 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:40.412335 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:40.412951 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:40.413083 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:40.417709 (+ 4626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:40.417716 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:40.417730 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:40.417769 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:40.420953 (+ 3184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:40.421657 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:40.421663 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:40.421669 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:40.421830 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:40.422270 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:40.422275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:40.422278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:40.423045 (+ 767us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:40.423384 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:40.423709 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 20:18:40.424167 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":399,"threads_started":1}
I20260809 20:18:40.430786 17287 raft_consensus.cc:348] T 00000000000000000000000000000000 P e06586bcc76f466eae83026c0c0d7491 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06586bcc76f466eae83026c0c0d7491" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 34143 } }
I20260809 20:18:40.431120 17287 raft_consensus.cc:374] T 00000000000000000000000000000000 P e06586bcc76f466eae83026c0c0d7491 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:40.431200 17287 raft_consensus.cc:729] T 00000000000000000000000000000000 P e06586bcc76f466eae83026c0c0d7491 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e06586bcc76f466eae83026c0c0d7491, State: Initialized, Role: FOLLOWER
I20260809 20:18:40.431742 17287 consensus_queue.cc:260] T 00000000000000000000000000000000 P e06586bcc76f466eae83026c0c0d7491 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06586bcc76f466eae83026c0c0d7491" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 34143 } }
I20260809 20:18:40.431946 17287 raft_consensus.cc:388] T 00000000000000000000000000000000 P e06586bcc76f466eae83026c0c0d7491 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:18:40.432041 17287 raft_consensus.cc:482] T 00000000000000000000000000000000 P e06586bcc76f466eae83026c0c0d7491 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:18:40.432160 17287 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e06586bcc76f466eae83026c0c0d7491 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:40.434368 17287 raft_consensus.cc:504] T 00000000000000000000000000000000 P e06586bcc76f466eae83026c0c0d7491 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06586bcc76f466eae83026c0c0d7491" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 34143 } }
I20260809 20:18:40.434805 17287 leader_election.cc:302] T 00000000000000000000000000000000 P e06586bcc76f466eae83026c0c0d7491 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e06586bcc76f466eae83026c0c0d7491; no voters:
I20260809 20:18:40.435407 17287 leader_election.cc:288] T 00000000000000000000000000000000 P e06586bcc76f466eae83026c0c0d7491 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:18:40.435551 17292 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e06586bcc76f466eae83026c0c0d7491 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:18:40.436612 17292 raft_consensus.cc:686] T 00000000000000000000000000000000 P e06586bcc76f466eae83026c0c0d7491 [term 1 LEADER]: Becoming Leader. State: Replica: e06586bcc76f466eae83026c0c0d7491, State: Running, Role: LEADER
I20260809 20:18:40.437177 17292 consensus_queue.cc:237] T 00000000000000000000000000000000 P e06586bcc76f466eae83026c0c0d7491 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06586bcc76f466eae83026c0c0d7491" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 34143 } }
I20260809 20:18:40.437587 17287 sys_catalog.cc:564] T 00000000000000000000000000000000 P e06586bcc76f466eae83026c0c0d7491 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:18:40.440328 17295 sys_catalog.cc:455] T 00000000000000000000000000000000 P e06586bcc76f466eae83026c0c0d7491 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e06586bcc76f466eae83026c0c0d7491. Latest consensus state: current_term: 1 leader_uuid: "e06586bcc76f466eae83026c0c0d7491" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06586bcc76f466eae83026c0c0d7491" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 34143 } } }
I20260809 20:18:40.440308 17294 sys_catalog.cc:455] T 00000000000000000000000000000000 P e06586bcc76f466eae83026c0c0d7491 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e06586bcc76f466eae83026c0c0d7491" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06586bcc76f466eae83026c0c0d7491" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 34143 } } }
I20260809 20:18:40.440697 17295 sys_catalog.cc:458] T 00000000000000000000000000000000 P e06586bcc76f466eae83026c0c0d7491 [sys.catalog]: This master's current role is: LEADER
I20260809 20:18:40.440759 17294 sys_catalog.cc:458] T 00000000000000000000000000000000 P e06586bcc76f466eae83026c0c0d7491 [sys.catalog]: This master's current role is: LEADER
I20260809 20:18:40.442209 17298 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:18:40.446990 17298 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:18:40.456090 17298 catalog_manager.cc:1347] Generated new cluster ID: 8154a44e230d47cfaef59befe9e236ce
I20260809 20:18:40.456219 17298 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:18:40.475451 17298 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:18:40.476959 17298 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:18:40.488917 17298 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e06586bcc76f466eae83026c0c0d7491: Generated new TSK 0
I20260809 20:18:40.489728 17298 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.61.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
I20260809 20:18:40.568055 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:34143
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:34273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:18:40.908871 17316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:40.909135 17316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:40.909292 17316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:40.916378 17316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:40.916510 17316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:40.916723 17316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:18:40.926405 17316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:34273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:34143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:40.927680 17316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:40.929013 17316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:40.942795 17321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:40.943688 17324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:40.943724 17322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:40.944522 17316 server_base.cc:1034] running on GCE node
I20260809 20:18:40.945086 17316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:40.946033 17316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:40.947247 17316 hybrid_clock.cc:648] HybridClock initialized: now 1786306720947184 us; error 55 us; skew 500 ppm
I20260809 20:18:40.947521 17316 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:18:40 dist-test-slave-6k22 krb5kdc[17199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306720, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:18:40.951004 17316 init.cc:377] Logged in from keytab as kudu/127.5.61.1@KRBTEST.COM (short username kudu)
I20260809 20:18:40.952596 17316 webserver.cc:460] Webserver started at http://127.5.61.1:43571/ using document root <none> and password file <none>
I20260809 20:18:40.953318 17316 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:40.953476 17316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:40.953827 17316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:18:40.957046 17316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "506dd6ec4dd944c2b5c4d2d2a0fe860f"
format_stamp: "Formatted at 2026-08-09 20:18:40 on dist-test-slave-6k22"
server_key: "02375a2aeea029c072960bf9ae9b7d80"
server_key_iv: "281caeb33587f9fec8f2daf4259530c7"
server_key_version: "encryptionkey@0"
I20260809 20:18:40.957944 17316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "506dd6ec4dd944c2b5c4d2d2a0fe860f"
format_stamp: "Formatted at 2026-08-09 20:18:40 on dist-test-slave-6k22"
server_key: "02375a2aeea029c072960bf9ae9b7d80"
server_key_iv: "281caeb33587f9fec8f2daf4259530c7"
server_key_version: "encryptionkey@0"
I20260809 20:18:40.963284 17316 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 20:18:40.966854 17331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:40.968223 17316 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:18:40.968412 17316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "506dd6ec4dd944c2b5c4d2d2a0fe860f"
format_stamp: "Formatted at 2026-08-09 20:18:40 on dist-test-slave-6k22"
server_key: "02375a2aeea029c072960bf9ae9b7d80"
server_key_iv: "281caeb33587f9fec8f2daf4259530c7"
server_key_version: "encryptionkey@0"
I20260809 20:18:40.968612 17316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:41.022248 17316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:41.027060 17316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:41.027374 17316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:41.028388 17316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:18:41.029796 17316 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:18:41.029915 17316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:41.030071 17316 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:18:41.030148 17316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:41.065734 17316 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:38801
I20260809 20:18:41.065824 17462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:38801 every 8 connection(s)
I20260809 20:18:41.067044 17316 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:18:41.075106 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 17316
I20260809 20:18:41.075349 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:18:41.076133 5364 external_mini_cluster.cc:1428] Setting key 281d7000c48a03ea58bc21d384b157aa
Aug 09 20:18:41 dist-test-slave-6k22 krb5kdc[17199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306720, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:18:41.088133 17465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:41.069717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:42449 (local address 127.5.61.62:34143)
0809 20:18:41.070193 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:41.070200 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:41.071250 (+ 1050us) server_negotiation.cc:407] Connection header received
0809 20:18:41.073093 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:41.073097 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:41.073200 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:41.073303 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:41.074757 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:41.075953 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:41.077852 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:41.078262 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:41.078434 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:41.082259 (+ 3825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:41.082264 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:41.082267 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:41.082298 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:41.085066 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:41.085789 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:41.085796 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:41.085800 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:41.085925 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:41.086381 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:41.086386 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:41.086389 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:41.086643 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:41.086803 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:41.087494 (+ 691us) server_negotiation.cc:299] Negotiation successful
0809 20:18:41.087745 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":226,"threads_started":1}
I20260809 20:18:41.089059 17466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:41.070334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:34143 (local address 127.5.61.1:42449)
0809 20:18:41.071078 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:41.071121 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:41.072841 (+ 1720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:41.073499 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:41.073507 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:41.074036 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:41.074572 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:41.074588 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:41.076106 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:41.076110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:41.077560 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:41.077572 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:41.078527 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:41.078537 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:41.078785 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:41.079486 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:41.079509 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:41.082029 (+ 2520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:41.085216 (+ 3187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:41.085226 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:41.085241 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:41.085656 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:41.086045 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:41.086049 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:41.086053 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:41.086259 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:41.086777 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:41.086784 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:41.087125 (+ 341us) client_negotiation.cc:769] Sending connection context
0809 20:18:41.087390 (+ 265us) client_negotiation.cc:240] Negotiation successful
0809 20:18:41.087947 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":328,"threads_started":1}
I20260809 20:18:41.090646 17463 heartbeater.cc:347] Connected to a master server at 127.5.61.62:34143
I20260809 20:18:41.090988 17463 heartbeater.cc:464] Registering TS with master...
I20260809 20:18:41.091943 17463 heartbeater.cc:511] Master 127.5.61.62:34143 requested a full tablet report, sending...
I20260809 20:18:41.094314 17249 ts_manager.cc:194] Registered new tserver with Master: 506dd6ec4dd944c2b5c4d2d2a0fe860f (127.5.61.1:38801)
I20260809 20:18:41.096555 17249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.1@KRBTEST.COM'} at 127.5.61.1:42449
WARNING: no policy specified for kudu/127.5.61.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
I20260809 20:18:41.137450 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:34143
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:34273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:18:41.479097 17469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:41.479387 17469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:41.479610 17469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:41.486858 17469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:41.486964 17469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:41.487123 17469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:18:41.496747 17469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:34273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:34143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:41.497993 17469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:41.499331 17469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:41.512467 17475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:41.512663 17476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:41.512929 17478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:41.513664 17469 server_base.cc:1034] running on GCE node
I20260809 20:18:41.514228 17469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:41.515131 17469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:41.516330 17469 hybrid_clock.cc:648] HybridClock initialized: now 1786306721516291 us; error 31 us; skew 500 ppm
I20260809 20:18:41.516636 17469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:18:41 dist-test-slave-6k22 krb5kdc[17199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306721, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:18:41.519827 17469 init.cc:377] Logged in from keytab as kudu/127.5.61.2@KRBTEST.COM (short username kudu)
I20260809 20:18:41.521240 17469 webserver.cc:460] Webserver started at http://127.5.61.2:35723/ using document root <none> and password file <none>
I20260809 20:18:41.521914 17469 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:41.522068 17469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:41.522367 17469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:18:41.525233 17469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "806f359491154349b5b56cecb2211b4d"
format_stamp: "Formatted at 2026-08-09 20:18:41 on dist-test-slave-6k22"
server_key: "47282ceaa870832fc720956ca4e0905c"
server_key_iv: "a191036a2d3c34df3254560219f86bb5"
server_key_version: "encryptionkey@0"
I20260809 20:18:41.526024 17469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "806f359491154349b5b56cecb2211b4d"
format_stamp: "Formatted at 2026-08-09 20:18:41 on dist-test-slave-6k22"
server_key: "47282ceaa870832fc720956ca4e0905c"
server_key_iv: "a191036a2d3c34df3254560219f86bb5"
server_key_version: "encryptionkey@0"
I20260809 20:18:41.531170 17469 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 20:18:41.534543 17489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:41.535934 17469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 20:18:41.536109 17469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "806f359491154349b5b56cecb2211b4d"
format_stamp: "Formatted at 2026-08-09 20:18:41 on dist-test-slave-6k22"
server_key: "47282ceaa870832fc720956ca4e0905c"
server_key_iv: "a191036a2d3c34df3254560219f86bb5"
server_key_version: "encryptionkey@0"
I20260809 20:18:41.536319 17469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:41.560701 17469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:41.564575 17469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:41.564880 17469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:41.565833 17469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:18:41.567157 17469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:18:41.567261 17469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:41.567418 17469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:18:41.567492 17469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:41.601147 17469 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:46749
I20260809 20:18:41.601203 17614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:46749 every 8 connection(s)
I20260809 20:18:41.602447 17469 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:18:41.603639 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 17469
I20260809 20:18:41.603842 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:18:41.604435 5364 external_mini_cluster.cc:1428] Setting key 6d0206c0825aa905ed0abf468ecaba76
Aug 09 20:18:41 dist-test-slave-6k22 krb5kdc[17199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306721, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:18:41.622928 17617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:41.605278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:50933 (local address 127.5.61.62:34143)
0809 20:18:41.605790 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:41.605798 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:41.606897 (+ 1099us) server_negotiation.cc:407] Connection header received
0809 20:18:41.608811 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:41.608817 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:41.608902 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:41.609019 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:41.610493 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:41.611191 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:41.612763 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:41.613180 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:41.613321 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:41.616826 (+ 3505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:41.616831 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:41.616834 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:41.616866 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:41.619608 (+ 2742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:41.620562 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:41.620566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:41.620568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:41.620692 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:41.621190 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:41.621193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:41.621195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:41.621455 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:41.621563 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:41.622487 (+ 924us) server_negotiation.cc:299] Negotiation successful
0809 20:18:41.622660 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":247,"threads_started":1}
I20260809 20:18:41.624462 17619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:41.605885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:34143 (local address 127.5.61.2:50933)
0809 20:18:41.606730 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:41.606774 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:41.608538 (+ 1764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:41.609249 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:41.609257 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:41.609767 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:41.610320 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:41.610337 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:41.611350 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:41.611354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:41.612585 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:41.612597 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:41.613419 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:41.613426 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:41.613650 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:41.614341 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:41.614364 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:41.616595 (+ 2231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:41.619743 (+ 3148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:41.619755 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:41.619773 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:41.620438 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:41.620799 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:41.620806 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:41.620813 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:41.621062 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:41.621606 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:41.621625 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:41.622087 (+ 462us) client_negotiation.cc:769] Sending connection context
0809 20:18:41.622393 (+ 306us) client_negotiation.cc:240] Negotiation successful
0809 20:18:41.622959 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":364,"threads_started":1}
I20260809 20:18:41.625936 17615 heartbeater.cc:347] Connected to a master server at 127.5.61.62:34143
I20260809 20:18:41.626284 17615 heartbeater.cc:464] Registering TS with master...
I20260809 20:18:41.627202 17615 heartbeater.cc:511] Master 127.5.61.62:34143 requested a full tablet report, sending...
I20260809 20:18:41.628880 17249 ts_manager.cc:194] Registered new tserver with Master: 806f359491154349b5b56cecb2211b4d (127.5.61.2:46749)
I20260809 20:18:41.630187 17249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.2@KRBTEST.COM'} at 127.5.61.2:50933
WARNING: no policy specified for kudu/127.5.61.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
I20260809 20:18:41.666752 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:34143
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:34273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:18:42.004946 17622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:18:42.005225 17622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:18:42.005319 17622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:18:42.012377 17622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:18:42.012477 17622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:18:42.012593 17622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:18:42.022317 17622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:34273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:34143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:18:42.023550 17622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:18:42.024930 17622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:18:42.038815 17627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:42.038785 17628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:18:42.038785 17630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:18:42.039882 17622 server_base.cc:1034] running on GCE node
I20260809 20:18:42.040545 17622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:18:42.041577 17622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:18:42.042802 17622 hybrid_clock.cc:648] HybridClock initialized: now 1786306722042769 us; error 52 us; skew 500 ppm
I20260809 20:18:42.043159 17622 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:18:42 dist-test-slave-6k22 krb5kdc[17199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306722, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:18:42.046469 17622 init.cc:377] Logged in from keytab as kudu/127.5.61.3@KRBTEST.COM (short username kudu)
I20260809 20:18:42.047971 17622 webserver.cc:460] Webserver started at http://127.5.61.3:38511/ using document root <none> and password file <none>
I20260809 20:18:42.048655 17622 fs_manager.cc:359] Metadata directory not provided
I20260809 20:18:42.048799 17622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:18:42.049095 17622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:18:42.052069 17622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "31c40c10db1e4e33b7fa4b1af8660c00"
format_stamp: "Formatted at 2026-08-09 20:18:42 on dist-test-slave-6k22"
server_key: "f936e25f104e9ed91312052f2f8be255"
server_key_iv: "2fbd93c69f8dd6149c9f0bf6b46c1adb"
server_key_version: "encryptionkey@0"
I20260809 20:18:42.052868 17622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "31c40c10db1e4e33b7fa4b1af8660c00"
format_stamp: "Formatted at 2026-08-09 20:18:42 on dist-test-slave-6k22"
server_key: "f936e25f104e9ed91312052f2f8be255"
server_key_iv: "2fbd93c69f8dd6149c9f0bf6b46c1adb"
server_key_version: "encryptionkey@0"
I20260809 20:18:42.057772 17622 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 20:18:42.061210 17639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:42.062508 17622 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 20:18:42.062690 17622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "31c40c10db1e4e33b7fa4b1af8660c00"
format_stamp: "Formatted at 2026-08-09 20:18:42 on dist-test-slave-6k22"
server_key: "f936e25f104e9ed91312052f2f8be255"
server_key_iv: "2fbd93c69f8dd6149c9f0bf6b46c1adb"
server_key_version: "encryptionkey@0"
I20260809 20:18:42.062909 17622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:18:42.080817 17622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:18:42.084611 17622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:18:42.084910 17622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:18:42.085867 17622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:18:42.087159 17622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:18:42.087280 17622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:42.087431 17622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:18:42.087519 17622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:18:42.100369 17463 heartbeater.cc:503] Master 127.5.61.62:34143 was elected leader, sending a full tablet report...
I20260809 20:18:42.121398 17622 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:33009
I20260809 20:18:42.121454 17769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:33009 every 8 connection(s)
I20260809 20:18:42.122653 17622 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:18:42.132386 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 17622
I20260809 20:18:42.132588 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:18:42.133035 5364 external_mini_cluster.cc:1428] Setting key d31cc8753a64b4f339382f0505a1c87f
Aug 09 20:18:42 dist-test-slave-6k22 krb5kdc[17199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306722, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:18:42.142629 17772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:42.125197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:49837 (local address 127.5.61.62:34143)
0809 20:18:42.125651 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:42.125656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:42.126616 (+ 960us) server_negotiation.cc:407] Connection header received
0809 20:18:42.128533 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:42.128539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:42.128628 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:42.128748 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:42.130233 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.131043 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.132687 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.133107 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.133248 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:42.136890 (+ 3642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:42.136894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:42.136898 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:42.136928 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:42.139955 (+ 3027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:42.140687 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:42.140695 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:42.140698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:42.140841 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:42.141215 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:42.141219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:42.141221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:42.141472 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:42.141581 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:42.142190 (+ 609us) server_negotiation.cc:299] Negotiation successful
0809 20:18:42.142361 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":217,"threads_started":1}
I20260809 20:18:42.143600 17773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:42.125732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:34143 (local address 127.5.61.3:49837)
0809 20:18:42.126427 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:42.126469 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:42.128206 (+ 1737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:42.128993 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:42.129000 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:42.129502 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:18:42.130061 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:42.130078 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.131215 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.131219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:42.132548 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:42.132561 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.133375 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.133385 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:42.133669 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:42.134390 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:18:42.134419 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:18:42.136712 (+ 2293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:18:42.140076 (+ 3364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:42.140083 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:42.140094 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:42.140544 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:42.140939 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:18:42.140942 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:18:42.140945 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:18:42.141116 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:18:42.141577 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:18:42.141583 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:18:42.141879 (+ 296us) client_negotiation.cc:769] Sending connection context
0809 20:18:42.142095 (+ 216us) client_negotiation.cc:240] Negotiation successful
0809 20:18:42.142459 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":277,"threads_started":1}
I20260809 20:18:42.145123 17770 heartbeater.cc:347] Connected to a master server at 127.5.61.62:34143
I20260809 20:18:42.145414 17770 heartbeater.cc:464] Registering TS with master...
I20260809 20:18:42.146152 17770 heartbeater.cc:511] Master 127.5.61.62:34143 requested a full tablet report, sending...
I20260809 20:18:42.147539 17249 ts_manager.cc:194] Registered new tserver with Master: 31c40c10db1e4e33b7fa4b1af8660c00 (127.5.61.3:33009)
I20260809 20:18:42.148386 17249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.3@KRBTEST.COM'} at 127.5.61.3:49837
I20260809 20:18:42.148600 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 20:18:42.166503 17772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:42.153985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38216 (local address 127.5.61.62:34143)
0809 20:18:42.154154 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:42.154159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:42.154699 (+ 540us) server_negotiation.cc:407] Connection header received
0809 20:18:42.154913 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:42.154918 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:42.154976 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:42.155081 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:42.156541 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.157246 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.158549 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.158956 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.159093 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:42.161398 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:42.161405 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:42.161409 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:42.161447 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:42.163852 (+ 2405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:42.164494 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:42.164501 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:42.164504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:42.164625 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:42.165027 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:42.165033 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:42.165037 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:42.165358 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:42.165614 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:42.165962 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 20:18:42.166181 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 20:18:42.170393 17249 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38216:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:18:42.180143 17249 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:18:42.203389 17786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:42.192349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46950 (local address 127.5.61.2:46749)
0809 20:18:42.193249 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:42.193256 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:42.193778 (+ 522us) server_negotiation.cc:407] Connection header received
0809 20:18:42.194049 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:42.194056 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:42.194300 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:42.194493 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:42.196894 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.198403 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.200842 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.202044 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.202179 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:42.202358 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:42.202896 (+ 538us) server_negotiation.cc:299] Negotiation successful
0809 20:18:42.203047 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":1135,"threads_started":1}
I20260809 20:18:42.203845 17785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:42.191947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39814 (local address 127.5.61.1:38801)
0809 20:18:42.192821 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:42.192832 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:42.193521 (+ 689us) server_negotiation.cc:407] Connection header received
0809 20:18:42.193960 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:42.193968 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:42.194236 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:42.194437 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:42.196142 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.197730 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.200459 (+ 2729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.201798 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.201944 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:42.202170 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:42.203329 (+ 1159us) server_negotiation.cc:299] Negotiation successful
0809 20:18:42.203505 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":450,"threads_started":1}
I20260809 20:18:42.204164 17783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:42.191782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:46749 (local address 127.0.0.1:46950)
0809 20:18:42.193651 (+ 1869us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:42.193670 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:42.193822 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:42.194875 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:42.194884 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:42.194919 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:42.196718 (+ 1799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:42.196732 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.198555 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.198563 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:42.200693 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:42.200702 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.202337 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.202343 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:42.202560 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:42.202705 (+ 145us) client_negotiation.cc:769] Sending connection context
0809 20:18:42.203669 (+ 964us) client_negotiation.cc:240] Negotiation successful
0809 20:18:42.203802 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1754,"mutex_wait_us":121,"thread_start_us":1074,"threads_started":1}
I20260809 20:18:42.204402 17784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:42.191598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:38801 (local address 127.0.0.1:39814)
0809 20:18:42.193364 (+ 1766us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:42.193414 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:42.193681 (+ 267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:42.194825 (+ 1144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:42.194835 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:42.194874 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:42.195895 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:42.195912 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.197898 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.197906 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:42.200292 (+ 2386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:42.200308 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.202008 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.202016 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:42.202567 (+ 551us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:42.202688 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 20:18:42.202851 (+ 163us) client_negotiation.cc:240] Negotiation successful
0809 20:18:42.203017 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1419,"mutex_wait_us":108,"thread_start_us":1798,"threads_started":1}
I20260809 20:18:42.205942 17787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:42.193965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:33009 (local address 127.0.0.1:46650)
0809 20:18:42.196240 (+ 2275us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:42.196264 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:42.196397 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:42.199101 (+ 2704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:42.199106 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:42.199135 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:42.199700 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:42.199718 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.201530 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.201535 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:42.203504 (+ 1969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:42.203518 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.205177 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.205183 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:42.205353 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:42.205386 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 20:18:42.205474 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 20:18:42.205600 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2101,"thread_start_us":1887,"threads_started":1}
I20260809 20:18:42.206089 17788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:42.194087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46650 (local address 127.5.61.3:33009)
0809 20:18:42.198230 (+ 4143us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:42.198237 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:42.198264 (+ 27us) server_negotiation.cc:407] Connection header received
0809 20:18:42.198349 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:42.198356 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:42.198605 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:42.198795 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:42.199895 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.201376 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.203729 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.204977 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.205116 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:42.205331 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:42.205575 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 20:18:42.205737 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3983,"thread_start_us":3849,"threads_started":1}
I20260809 20:18:42.208146 17386 tablet_service.cc:1467] Processing CreateTablet for tablet 88577132a338401c8028ac80f1a4bae8 (DEFAULT_TABLE table=test-table [id=d02c50b7ecc54af6b9d7c83d6920a162]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:18:42.208257 17542 tablet_service.cc:1467] Processing CreateTablet for tablet 88577132a338401c8028ac80f1a4bae8 (DEFAULT_TABLE table=test-table [id=d02c50b7ecc54af6b9d7c83d6920a162]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:18:42.209429 17386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88577132a338401c8028ac80f1a4bae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:42.209613 17542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88577132a338401c8028ac80f1a4bae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:42.209441 17696 tablet_service.cc:1467] Processing CreateTablet for tablet 88577132a338401c8028ac80f1a4bae8 (DEFAULT_TABLE table=test-table [id=d02c50b7ecc54af6b9d7c83d6920a162]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:18:42.210719 17696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88577132a338401c8028ac80f1a4bae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:18:42.221220 17789 tablet_bootstrap.cc:492] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f: Bootstrap starting.
I20260809 20:18:42.224655 17790 tablet_bootstrap.cc:492] T 88577132a338401c8028ac80f1a4bae8 P 31c40c10db1e4e33b7fa4b1af8660c00: Bootstrap starting.
I20260809 20:18:42.225486 17791 tablet_bootstrap.cc:492] T 88577132a338401c8028ac80f1a4bae8 P 806f359491154349b5b56cecb2211b4d: Bootstrap starting.
I20260809 20:18:42.225880 17789 tablet_bootstrap.cc:654] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:42.227309 17789 log.cc:824] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f: Log is configured to *not* fsync() on all Append() calls
I20260809 20:18:42.228317 17790 tablet_bootstrap.cc:654] T 88577132a338401c8028ac80f1a4bae8 P 31c40c10db1e4e33b7fa4b1af8660c00: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:42.228703 17791 tablet_bootstrap.cc:654] T 88577132a338401c8028ac80f1a4bae8 P 806f359491154349b5b56cecb2211b4d: Neither blocks nor log segments found. Creating new log.
I20260809 20:18:42.229704 17790 log.cc:824] T 88577132a338401c8028ac80f1a4bae8 P 31c40c10db1e4e33b7fa4b1af8660c00: Log is configured to *not* fsync() on all Append() calls
I20260809 20:18:42.229920 17791 log.cc:824] T 88577132a338401c8028ac80f1a4bae8 P 806f359491154349b5b56cecb2211b4d: Log is configured to *not* fsync() on all Append() calls
I20260809 20:18:42.230984 17789 tablet_bootstrap.cc:492] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f: No bootstrap required, opened a new log
I20260809 20:18:42.231323 17789 ts_tablet_manager.cc:1397] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260809 20:18:42.232846 17791 tablet_bootstrap.cc:492] T 88577132a338401c8028ac80f1a4bae8 P 806f359491154349b5b56cecb2211b4d: No bootstrap required, opened a new log
I20260809 20:18:42.233103 17791 ts_tablet_manager.cc:1397] T 88577132a338401c8028ac80f1a4bae8 P 806f359491154349b5b56cecb2211b4d: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 20:18:42.238098 17790 tablet_bootstrap.cc:492] T 88577132a338401c8028ac80f1a4bae8 P 31c40c10db1e4e33b7fa4b1af8660c00: No bootstrap required, opened a new log
I20260809 20:18:42.238484 17790 ts_tablet_manager.cc:1397] T 88577132a338401c8028ac80f1a4bae8 P 31c40c10db1e4e33b7fa4b1af8660c00: Time spent bootstrapping tablet: real 0.014s user 0.013s sys 0.000s
I20260809 20:18:42.241118 17791 raft_consensus.cc:348] T 88577132a338401c8028ac80f1a4bae8 P 806f359491154349b5b56cecb2211b4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506dd6ec4dd944c2b5c4d2d2a0fe860f" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 38801 } } peers { permanent_uuid: "806f359491154349b5b56cecb2211b4d" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 46749 } } peers { permanent_uuid: "31c40c10db1e4e33b7fa4b1af8660c00" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33009 } }
I20260809 20:18:42.241503 17791 raft_consensus.cc:374] T 88577132a338401c8028ac80f1a4bae8 P 806f359491154349b5b56cecb2211b4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:42.241626 17791 raft_consensus.cc:729] T 88577132a338401c8028ac80f1a4bae8 P 806f359491154349b5b56cecb2211b4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 806f359491154349b5b56cecb2211b4d, State: Initialized, Role: FOLLOWER
I20260809 20:18:42.242226 17791 consensus_queue.cc:260] T 88577132a338401c8028ac80f1a4bae8 P 806f359491154349b5b56cecb2211b4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506dd6ec4dd944c2b5c4d2d2a0fe860f" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 38801 } } peers { permanent_uuid: "806f359491154349b5b56cecb2211b4d" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 46749 } } peers { permanent_uuid: "31c40c10db1e4e33b7fa4b1af8660c00" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33009 } }
I20260809 20:18:42.242919 17789 raft_consensus.cc:348] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506dd6ec4dd944c2b5c4d2d2a0fe860f" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 38801 } } peers { permanent_uuid: "806f359491154349b5b56cecb2211b4d" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 46749 } } peers { permanent_uuid: "31c40c10db1e4e33b7fa4b1af8660c00" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33009 } }
I20260809 20:18:42.243364 17615 heartbeater.cc:503] Master 127.5.61.62:34143 was elected leader, sending a full tablet report...
I20260809 20:18:42.243391 17789 raft_consensus.cc:374] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:42.243491 17789 raft_consensus.cc:729] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506dd6ec4dd944c2b5c4d2d2a0fe860f, State: Initialized, Role: FOLLOWER
I20260809 20:18:42.244002 17791 ts_tablet_manager.cc:1428] T 88577132a338401c8028ac80f1a4bae8 P 806f359491154349b5b56cecb2211b4d: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260809 20:18:42.244167 17789 consensus_queue.cc:260] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506dd6ec4dd944c2b5c4d2d2a0fe860f" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 38801 } } peers { permanent_uuid: "806f359491154349b5b56cecb2211b4d" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 46749 } } peers { permanent_uuid: "31c40c10db1e4e33b7fa4b1af8660c00" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33009 } }
I20260809 20:18:42.245834 17789 ts_tablet_manager.cc:1428] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260809 20:18:42.251297 17790 raft_consensus.cc:348] T 88577132a338401c8028ac80f1a4bae8 P 31c40c10db1e4e33b7fa4b1af8660c00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506dd6ec4dd944c2b5c4d2d2a0fe860f" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 38801 } } peers { permanent_uuid: "806f359491154349b5b56cecb2211b4d" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 46749 } } peers { permanent_uuid: "31c40c10db1e4e33b7fa4b1af8660c00" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33009 } }
I20260809 20:18:42.251654 17790 raft_consensus.cc:374] T 88577132a338401c8028ac80f1a4bae8 P 31c40c10db1e4e33b7fa4b1af8660c00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:18:42.251749 17790 raft_consensus.cc:729] T 88577132a338401c8028ac80f1a4bae8 P 31c40c10db1e4e33b7fa4b1af8660c00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31c40c10db1e4e33b7fa4b1af8660c00, State: Initialized, Role: FOLLOWER
I20260809 20:18:42.252223 17790 consensus_queue.cc:260] T 88577132a338401c8028ac80f1a4bae8 P 31c40c10db1e4e33b7fa4b1af8660c00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506dd6ec4dd944c2b5c4d2d2a0fe860f" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 38801 } } peers { permanent_uuid: "806f359491154349b5b56cecb2211b4d" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 46749 } } peers { permanent_uuid: "31c40c10db1e4e33b7fa4b1af8660c00" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33009 } }
I20260809 20:18:42.253139 17770 heartbeater.cc:503] Master 127.5.61.62:34143 was elected leader, sending a full tablet report...
I20260809 20:18:42.253661 17790 ts_tablet_manager.cc:1428] T 88577132a338401c8028ac80f1a4bae8 P 31c40c10db1e4e33b7fa4b1af8660c00: Time spent starting tablet: real 0.015s user 0.011s sys 0.000s
I20260809 20:18:42.316694 17796 raft_consensus.cc:482] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:18:42.317018 17796 raft_consensus.cc:504] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506dd6ec4dd944c2b5c4d2d2a0fe860f" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 38801 } } peers { permanent_uuid: "806f359491154349b5b56cecb2211b4d" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 46749 } } peers { permanent_uuid: "31c40c10db1e4e33b7fa4b1af8660c00" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33009 } }
I20260809 20:18:42.318866 17796 leader_election.cc:288] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 806f359491154349b5b56cecb2211b4d (127.5.61.2:46749), 31c40c10db1e4e33b7fa4b1af8660c00 (127.5.61.3:33009)
W20260809 20:18:42.319689 17464 tablet.cc:2369] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:18:42.326215 17786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:42.319187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:51879 (local address 127.5.61.2:46749)
0809 20:18:42.319452 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:42.319461 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:42.320315 (+ 854us) server_negotiation.cc:407] Connection header received
0809 20:18:42.320633 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:42.320639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:42.320714 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:42.320782 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:42.321405 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.322362 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.324368 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.325233 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.325338 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:42.325416 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:42.325838 (+ 422us) server_negotiation.cc:299] Negotiation successful
0809 20:18:42.325968 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 20:18:42.326295 17800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:42.319122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:46749 (local address 127.5.61.1:51879)
0809 20:18:42.320181 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:42.320207 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:42.320442 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:42.320829 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:42.320836 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:42.320866 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:42.321254 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:42.321267 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.322492 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.322497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:42.324197 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:42.324210 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.325455 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.325462 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:42.325650 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:42.325685 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 20:18:42.325792 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 20:18:42.325885 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"spinlock_wait_cycles":22784,"thread_start_us":804,"threads_started":1}
I20260809 20:18:42.326480 17788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:42.319485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:54825 (local address 127.5.61.3:33009)
0809 20:18:42.319759 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:42.319768 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:42.320399 (+ 631us) server_negotiation.cc:407] Connection header received
0809 20:18:42.320703 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:42.320708 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:42.320795 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:42.320885 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:18:42.321582 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.322634 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.324657 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.325484 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.325620 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:42.325724 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:42.326053 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 20:18:42.326173 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 20:18:42.326751 17799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:42.319123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:33009 (local address 127.5.61.1:54825)
0809 20:18:42.320285 (+ 1162us) negotiation.cc:107] Waiting for socket to connect
0809 20:18:42.320303 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 20:18:42.320520 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:18:42.320929 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:18:42.320935 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:18:42.320968 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:18:42.321350 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:42.321370 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.322805 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.322809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:42.324500 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:18:42.324514 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.325699 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.325703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:18:42.325883 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:42.325914 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 20:18:42.326012 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 20:18:42.326133 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1034,"mutex_wait_us":62,"thread_start_us":1082,"threads_started":1}
I20260809 20:18:42.327334 17566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88577132a338401c8028ac80f1a4bae8" candidate_uuid: "506dd6ec4dd944c2b5c4d2d2a0fe860f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "806f359491154349b5b56cecb2211b4d" is_pre_election: true
I20260809 20:18:42.327688 17718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88577132a338401c8028ac80f1a4bae8" candidate_uuid: "506dd6ec4dd944c2b5c4d2d2a0fe860f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31c40c10db1e4e33b7fa4b1af8660c00" is_pre_election: true
I20260809 20:18:42.327834 17566 raft_consensus.cc:2454] T 88577132a338401c8028ac80f1a4bae8 P 806f359491154349b5b56cecb2211b4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 506dd6ec4dd944c2b5c4d2d2a0fe860f in term 0.
I20260809 20:18:42.328173 17718 raft_consensus.cc:2454] T 88577132a338401c8028ac80f1a4bae8 P 31c40c10db1e4e33b7fa4b1af8660c00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 506dd6ec4dd944c2b5c4d2d2a0fe860f in term 0.
I20260809 20:18:42.328500 17334 leader_election.cc:302] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 506dd6ec4dd944c2b5c4d2d2a0fe860f, 806f359491154349b5b56cecb2211b4d; no voters:
I20260809 20:18:42.328986 17796 raft_consensus.cc:2781] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:18:42.329126 17796 raft_consensus.cc:482] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:18:42.329244 17796 raft_consensus.cc:3037] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:42.331483 17796 raft_consensus.cc:504] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506dd6ec4dd944c2b5c4d2d2a0fe860f" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 38801 } } peers { permanent_uuid: "806f359491154349b5b56cecb2211b4d" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 46749 } } peers { permanent_uuid: "31c40c10db1e4e33b7fa4b1af8660c00" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33009 } }
I20260809 20:18:42.332139 17796 leader_election.cc:288] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f [CANDIDATE]: Term 1 election: Requested vote from peers 806f359491154349b5b56cecb2211b4d (127.5.61.2:46749), 31c40c10db1e4e33b7fa4b1af8660c00 (127.5.61.3:33009)
I20260809 20:18:42.332628 17566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88577132a338401c8028ac80f1a4bae8" candidate_uuid: "506dd6ec4dd944c2b5c4d2d2a0fe860f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "806f359491154349b5b56cecb2211b4d"
I20260809 20:18:42.332666 17718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88577132a338401c8028ac80f1a4bae8" candidate_uuid: "506dd6ec4dd944c2b5c4d2d2a0fe860f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31c40c10db1e4e33b7fa4b1af8660c00"
I20260809 20:18:42.332818 17566 raft_consensus.cc:3037] T 88577132a338401c8028ac80f1a4bae8 P 806f359491154349b5b56cecb2211b4d [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:42.332876 17718 raft_consensus.cc:3037] T 88577132a338401c8028ac80f1a4bae8 P 31c40c10db1e4e33b7fa4b1af8660c00 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:18:42.335045 17566 raft_consensus.cc:2454] T 88577132a338401c8028ac80f1a4bae8 P 806f359491154349b5b56cecb2211b4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 506dd6ec4dd944c2b5c4d2d2a0fe860f in term 1.
I20260809 20:18:42.335050 17718 raft_consensus.cc:2454] T 88577132a338401c8028ac80f1a4bae8 P 31c40c10db1e4e33b7fa4b1af8660c00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 506dd6ec4dd944c2b5c4d2d2a0fe860f in term 1.
I20260809 20:18:42.335476 17335 leader_election.cc:302] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31c40c10db1e4e33b7fa4b1af8660c00, 506dd6ec4dd944c2b5c4d2d2a0fe860f; no voters:
I20260809 20:18:42.335783 17796 raft_consensus.cc:2781] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:18:42.336279 17796 raft_consensus.cc:686] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f [term 1 LEADER]: Becoming Leader. State: Replica: 506dd6ec4dd944c2b5c4d2d2a0fe860f, State: Running, Role: LEADER
I20260809 20:18:42.336735 17796 consensus_queue.cc:237] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506dd6ec4dd944c2b5c4d2d2a0fe860f" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 38801 } } peers { permanent_uuid: "806f359491154349b5b56cecb2211b4d" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 46749 } } peers { permanent_uuid: "31c40c10db1e4e33b7fa4b1af8660c00" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33009 } }
I20260809 20:18:42.341492 17249 catalog_manager.cc:5495] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f reported cstate change: term changed from 0 to 1, leader changed from <none> to 506dd6ec4dd944c2b5c4d2d2a0fe860f (127.5.61.1). New cstate: current_term: 1 leader_uuid: "506dd6ec4dd944c2b5c4d2d2a0fe860f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506dd6ec4dd944c2b5c4d2d2a0fe860f" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 38801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "806f359491154349b5b56cecb2211b4d" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 46749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31c40c10db1e4e33b7fa4b1af8660c00" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33009 } health_report { overall_health: UNKNOWN } } }
W20260809 20:18:42.354511 17616 tablet.cc:2369] T 88577132a338401c8028ac80f1a4bae8 P 806f359491154349b5b56cecb2211b4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:18:42.363656 17785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:42.358744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39824 (local address 127.5.61.1:38801)
0809 20:18:42.358923 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:42.358928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:42.358944 (+ 16us) server_negotiation.cc:407] Connection header received
0809 20:18:42.359104 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:42.359107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:42.359189 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:42.359320 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:18:42.359770 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.360489 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.361783 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.362170 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.362259 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:42.362679 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:18:42.363158 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:18:42.363273 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:42.363339 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 20:18:42.363403 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 20:18:42.371627 17718 raft_consensus.cc:1261] T 88577132a338401c8028ac80f1a4bae8 P 31c40c10db1e4e33b7fa4b1af8660c00 [term 1 FOLLOWER]: Refusing update from remote peer 506dd6ec4dd944c2b5c4d2d2a0fe860f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:18:42.371701 17566 raft_consensus.cc:1261] T 88577132a338401c8028ac80f1a4bae8 P 806f359491154349b5b56cecb2211b4d [term 1 FOLLOWER]: Refusing update from remote peer 506dd6ec4dd944c2b5c4d2d2a0fe860f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:18:42.372798 17801 consensus_queue.cc:1035] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f [LEADER]: Connected to new peer: Peer: permanent_uuid: "31c40c10db1e4e33b7fa4b1af8660c00" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:18:42.373159 17796 consensus_queue.cc:1035] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f [LEADER]: Connected to new peer: Peer: permanent_uuid: "806f359491154349b5b56cecb2211b4d" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 46749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 20:18:42.374329 17771 tablet.cc:2369] T 88577132a338401c8028ac80f1a4bae8 P 31c40c10db1e4e33b7fa4b1af8660c00: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:18:42.388324 17804 mvcc.cc:205] Tried to move back new op lower bound from 7316712334824411136 to 7316712334695231488. Current Snapshot: MvccSnapshot[applied={T|T < 7316712334824411136}]
I20260809 20:18:42.408176 17249 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:18:42.408439 17249 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:18:42.412603 17249 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e06586bcc76f466eae83026c0c0d7491: Sending DeleteTablet for 3 replicas of tablet 88577132a338401c8028ac80f1a4bae8
I20260809 20:18:42.413842 17696 tablet_service.cc:1514] Processing DeleteTablet for tablet 88577132a338401c8028ac80f1a4bae8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:18:42 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:46650
I20260809 20:18:42.413918 17386 tablet_service.cc:1514] Processing DeleteTablet for tablet 88577132a338401c8028ac80f1a4bae8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:18:42 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:39814
I20260809 20:18:42.415144 17542 tablet_service.cc:1514] Processing DeleteTablet for tablet 88577132a338401c8028ac80f1a4bae8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:18:42 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:46950
I20260809 20:18:42.417243 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 17316
I20260809 20:18:42.425143 17815 tablet_replica.cc:330] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f: stopping tablet replica
I20260809 20:18:42.425874 17815 raft_consensus.cc:2229] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:18:42.426960 17815 raft_consensus.cc:2258] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:18:42.429490 17814 tablet_replica.cc:330] T 88577132a338401c8028ac80f1a4bae8 P 31c40c10db1e4e33b7fa4b1af8660c00: stopping tablet replica
I20260809 20:18:42.429848 17814 raft_consensus.cc:2229] T 88577132a338401c8028ac80f1a4bae8 P 31c40c10db1e4e33b7fa4b1af8660c00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:18:42.430266 17814 raft_consensus.cc:2258] T 88577132a338401c8028ac80f1a4bae8 P 31c40c10db1e4e33b7fa4b1af8660c00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:18:42.431219 17815 ts_tablet_manager.cc:1905] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:18:42.433269 17814 ts_tablet_manager.cc:1905] T 88577132a338401c8028ac80f1a4bae8 P 31c40c10db1e4e33b7fa4b1af8660c00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:18:42.439293 17815 ts_tablet_manager.cc:1918] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:18:42.439535 17815 log.cc:1196] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/88577132a338401c8028ac80f1a4bae8
I20260809 20:18:42.440183 17814 ts_tablet_manager.cc:1918] T 88577132a338401c8028ac80f1a4bae8 P 31c40c10db1e4e33b7fa4b1af8660c00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:18:42.440373 17814 log.cc:1196] T 88577132a338401c8028ac80f1a4bae8 P 31c40c10db1e4e33b7fa4b1af8660c00: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/88577132a338401c8028ac80f1a4bae8
I20260809 20:18:42.440992 17815 ts_tablet_manager.cc:1939] T 88577132a338401c8028ac80f1a4bae8 P 506dd6ec4dd944c2b5c4d2d2a0fe860f: Deleting consensus metadata
I20260809 20:18:42.440985 17814 ts_tablet_manager.cc:1939] T 88577132a338401c8028ac80f1a4bae8 P 31c40c10db1e4e33b7fa4b1af8660c00: Deleting consensus metadata
I20260809 20:18:42.442741 17234 catalog_manager.cc:4841] TS 31c40c10db1e4e33b7fa4b1af8660c00 (127.5.61.3:33009): tablet 88577132a338401c8028ac80f1a4bae8 (table test-table [id=d02c50b7ecc54af6b9d7c83d6920a162]) successfully deleted
I20260809 20:18:42.443094 17234 catalog_manager.cc:4841] TS 506dd6ec4dd944c2b5c4d2d2a0fe860f (127.5.61.1:38801): tablet 88577132a338401c8028ac80f1a4bae8 (table test-table [id=d02c50b7ecc54af6b9d7c83d6920a162]) successfully deleted
I20260809 20:18:42.443399 17816 tablet_replica.cc:330] T 88577132a338401c8028ac80f1a4bae8 P 806f359491154349b5b56cecb2211b4d: stopping tablet replica
I20260809 20:18:42.443845 17816 raft_consensus.cc:2229] T 88577132a338401c8028ac80f1a4bae8 P 806f359491154349b5b56cecb2211b4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:18:42.444376 17816 raft_consensus.cc:2258] T 88577132a338401c8028ac80f1a4bae8 P 806f359491154349b5b56cecb2211b4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:18:42.446323 17816 ts_tablet_manager.cc:1905] T 88577132a338401c8028ac80f1a4bae8 P 806f359491154349b5b56cecb2211b4d: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 20:18:42 dist-test-slave-6k22 krb5kdc[17199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.1@KRBTEST.COM
I20260809 20:18:42.452827 17816 ts_tablet_manager.cc:1918] T 88577132a338401c8028ac80f1a4bae8 P 806f359491154349b5b56cecb2211b4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:18:42.452993 17816 log.cc:1196] T 88577132a338401c8028ac80f1a4bae8 P 806f359491154349b5b56cecb2211b4d: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/88577132a338401c8028ac80f1a4bae8
I20260809 20:18:42.453537 17816 ts_tablet_manager.cc:1939] T 88577132a338401c8028ac80f1a4bae8 P 806f359491154349b5b56cecb2211b4d: Deleting consensus metadata
I20260809 20:18:42.455426 17233 catalog_manager.cc:4841] TS 806f359491154349b5b56cecb2211b4d (127.5.61.2:46749): tablet 88577132a338401c8028ac80f1a4bae8 (table test-table [id=d02c50b7ecc54af6b9d7c83d6920a162]) successfully deleted
I20260809 20:18:42.458850 17785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:42.418850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39830 (local address 127.5.61.1:38801)
0809 20:18:42.419037 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:42.419044 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:42.440271 (+ 21227us) server_negotiation.cc:407] Connection header received
0809 20:18:42.440502 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:42.440510 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:42.440599 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:42.440694 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:42.442648 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.443946 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.445794 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.446532 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.446663 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:42.451904 (+ 5241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:42.451918 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:42.451936 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:42.451986 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:42.456082 (+ 4096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:42.456855 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:42.456861 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:42.456863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:42.456996 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:42.457380 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:42.457382 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:42.457384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:42.457702 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:42.457922 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:42.458300 (+ 378us) server_negotiation.cc:299] Negotiation successful
0809 20:18:42.458508 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 20:18:42.459532 17458 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:42.559583 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 17316
I20260809 20:18:42.580420 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 17469
Aug 09 20:18:42 dist-test-slave-6k22 krb5kdc[17199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.2@KRBTEST.COM
I20260809 20:18:42.595525 17786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:42.581197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46954 (local address 127.5.61.2:46749)
0809 20:18:42.581379 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:42.581384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:42.581404 (+ 20us) server_negotiation.cc:407] Connection header received
0809 20:18:42.581574 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:42.581579 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:42.581641 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:42.581726 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:42.582909 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.583750 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.584919 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.585295 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.585396 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:42.589534 (+ 4138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:42.589547 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:42.589561 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:42.589594 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:42.592779 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:42.593445 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:42.593454 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:42.593458 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:42.593603 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:42.593986 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:42.593992 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:42.593996 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:42.594375 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:42.594677 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:42.594986 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 20:18:42.595200 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 20:18:42.596253 17609 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:42.686316 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 17469
I20260809 20:18:42.707649 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 17622
Aug 09 20:18:42 dist-test-slave-6k22 krb5kdc[17199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.3@KRBTEST.COM
I20260809 20:18:42.727461 17788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:18:42.708427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46666 (local address 127.5.61.3:33009)
0809 20:18:42.708620 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 20:18:42.708628 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:18:42.708652 (+ 24us) server_negotiation.cc:407] Connection header received
0809 20:18:42.708883 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:18:42.708887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:18:42.708953 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:18:42.709072 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:18:42.710399 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.711125 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.712489 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:18:42.712866 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:18:42.712958 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:18:42.722483 (+ 9525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:18:42.722499 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:18:42.722516 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:18:42.722550 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:18:42.725191 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:42.725782 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:42.725788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:42.725791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:42.725921 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:18:42.726231 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:18:42.726234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:18:42.726236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:18:42.726582 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:18:42.726788 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 20:18:42.727039 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 20:18:42.727204 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 20:18:42.728088 17765 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:42.815394 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 17622
I20260809 20:18:42.837559 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 17215
I20260809 20:18:42.838774 17281 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:18:42.947927 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 17215
2026-08-09T20:18:42Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4448 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:18:43 dist-test-slave-6k22 krb5kdc[17826](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:18:43 dist-test-slave-6k22 krb5kdc[17826](info): set up 2 sockets
Aug 09 20:18:43 dist-test-slave-6k22 krb5kdc[17826](info): commencing operation
krb5kdc: starting...
W20260809 20:18:44.127111 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:18:44 dist-test-slave-6k22 krb5kdc[17826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:18:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:18:44Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.61.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.61.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.61.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/rangeradmin_127.5.61.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.61.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/rangeradmin_127.5.61.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.61.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.61.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.61.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/rangerlookup_127.5.61.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.61.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/rangerlookup_127.5.61.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.61.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.61.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.61.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/HTTP_127.5.61.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.61.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/HTTP_127.5.61.20@KRBTEST.COM.keytab.
I20260809 20:18:44.504909 5364 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task7uf_Cz/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/postgres -l logfile start
2026-08-09 20:18:47.024 UTC [17863] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 20:18:47.024 UTC [17863] LOG: listening on IPv4 address "127.5.61.20", port 51901
2026-08-09 20:18:47.028 UTC [17863] LOG: listening on Unix socket "/tmp/.s.PGSQL.51901"
2026-08-09 20:18:47.051 UTC [17866] LOG: database system was shut down at 2026-08-09 20:18:45 UTC
2026-08-09 20:18:47.054 UTC [17863] LOG: database system is ready to accept connections
I20260809 20:18:48.112751 5364 mini_postgres.cc:96] Postgres bound to 51901
2026-08-09 20:18:48.136 UTC [17875] FATAL: database "slave" does not exist
127.5.61.20:51901 - accepting connections
I20260809 20:18:48.137472 5364 mini_ranger.cc:161] Starting Ranger...
I20260809 20:18:48.167446 5364 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 20:18:48.645649 5364 mini_ranger.cc:87] Created ranger Postgres database
I20260809 20:18:48.645807 5364 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ranger-admin
2026-08-09 20:18:48,960 [I] DB FLAVOR :POSTGRES
2026-08-09 20:18:48,961 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 20:18:48,961 [I] Checking connection..
2026-08-09 20:18:48,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:51901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 20:18:49,217 [I] Checking connection passed.
2026-08-09 20:18:49,217 [I] --------- Verifying version history table ---------
2026-08-09 20:18:49,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:51901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 20:18:49,463 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 20:18:49,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:51901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 20:18:49,723 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 20:18:49,723 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 20:18:49,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:51901/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 20:18:49.976 UTC [17943] WARNING: there is no transaction in progress
2026-08-09 20:18:49,984 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 20:18:49,984 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:51901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 20:18:50,227 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 20:18:50,227 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 20:18:50,227 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:51901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 20:18:50,471 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:51901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 20:18:50,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:51901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6k22.c.gcp-upstream.internal','N') ;"
2026-08-09 20:18:50,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:51901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 20:18:51,198 [I] Table x_portal_user does not exist in database ranger
2026-08-09 20:18:51,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:51901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 20:18:51,453 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 20:18:51,454 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:51901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 20:18:51,709 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:51901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 20:18:51,967 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 20:18:51,967 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:51901/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 20:18:52.911 UTC [18065] WARNING: there is no transaction in progress
2026-08-09 20:18:52.926 UTC [18065] WARNING: there is no transaction in progress
2026-08-09 20:18:52.941 UTC [18065] WARNING: there is no transaction in progress
2026-08-09 20:18:52.955 UTC [18065] WARNING: there is no transaction in progress
2026-08-09 20:18:52.972 UTC [18065] WARNING: there is no transaction in progress
2026-08-09 20:18:52.990 UTC [18065] WARNING: there is no transaction in progress
2026-08-09 20:18:53.042 UTC [18065] WARNING: there is no transaction in progress
2026-08-09 20:18:53.049 UTC [18065] WARNING: there is no transaction in progress
2026-08-09 20:18:53.063 UTC [18065] WARNING: there is no transaction in progress
2026-08-09 20:18:53.072 UTC [18065] WARNING: there is no transaction in progress
2026-08-09 20:18:53.082 UTC [18065] WARNING: there is no transaction in progress
2026-08-09 20:18:53.095 UTC [18065] WARNING: there is no transaction in progress
2026-08-09 20:18:53.104 UTC [18065] WARNING: there is no transaction in progress
2026-08-09 20:18:53.110 UTC [18065] WARNING: there is no transaction in progress
2026-08-09 20:18:53.117 UTC [18065] WARNING: there is no transaction in progress
2026-08-09 20:18:53.123 UTC [18065] WARNING: there is no transaction in progress
2026-08-09 20:18:53,742 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 20:18:53,743 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:51901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 20:18:53,993 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 20:18:53,994 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:51901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 20:18:54,258 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 20:18:54,258 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:51901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 20:18:54,492 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 20:18:54,492 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:51901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 20:18:54,732 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:51901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 20:18:54,987 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:51901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6k22.c.gcp-upstream.internal';"
2026-08-09 20:18:55,233 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 20:18:55,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:51901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 20:18:55,502 [I] DB_PATCHES have already been applied
I20260809 20:18:55.508589 5364 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ranger-admin:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task7uf_Cz/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 20:18:55.508708 5364 mini_ranger.cc:193] Using host: 127.5.61.20
I20260809 20:18:55.529500 5364 mini_ranger.cc:227] Ranger admin URL: http://127.5.61.20:42767
Aug 09, 2026 8:18:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 8:18:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 8:18:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 8:18:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 8:18:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 8:18:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 8:18:56 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 20:18:56.740634 5364 mini_ranger.cc:160] Time spent starting Ranger: real 8.603s user 0.008s sys 0.049s
Aug 09, 2026 8:19:16 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 8:19:17 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 8:19:17 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 8:19:17 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 8:19:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 8:19:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 8:19:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 8:19:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 8:19:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 8:19:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 8:19:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 8:19:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 8:19:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 8:19:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 8:19:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 8:19:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 8:19:17 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 8:19:18 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 20:19:18.571856 5364 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.61.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.61.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.61.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/rangerkms_127.5.61.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.61.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/rangerkms_127.5.61.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.61.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.61.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.61.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/HTTP_127.5.61.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.61.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/HTTP_127.5.61.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 20:19:18 dist-test-slave-6k22 krb5kdc[17826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306758, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 20:19:18.775120 5364 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 20:19:18.804179 5364 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 20:19:19.658295 5364 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 20:19:19.658504 5364 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ranger-kms
I20260809 20:19:19.658628 5364 mini_ranger_kms.cc:227] Using postgres at 127.5.61.20:51901
2026-08-09 20:19:20,220 [I] DB FLAVOR :POSTGRES
2026-08-09 20:19:20,221 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 20:19:20,221 [I] Checking connection
2026-08-09 20:19:20,221 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:51901/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 20:19:20,486 [I] connection success
2026-08-09 20:19:20,486 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 20:19:20,486 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 20:19:20,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:51901/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 20:19:20,715 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 20:19:20,715 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 20:19:20,715 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 20:19:20,715 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:51901/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 20:19:20,981 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 20:19:21.443117 5364 mini_ranger_kms.cc:310] Created kms service
I20260809 20:19:21.635885 5364 mini_ranger_kms.cc:326] Created kudu user
I20260809 20:19:21.682058 5364 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 20:19:21 dist-test-slave-6k22 krb5kdc[17826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306758, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.61.20@KRBTEST.COM
I20260809 20:19:22.100533 5364 mini_ranger_kms.cc:384] Added ranger policy
I20260809 20:19:22.101925 5364 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ranger-kms:/tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 20:19:22.102011 5364 mini_ranger_kms.cc:240] Using host: 127.5.61.20
I20260809 20:19:22.128661 5364 mini_ranger_kms.cc:276] Ranger KMS PID: 18265
I20260809 20:19:22.128847 5364 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.61.20:51907
Aug 09, 2026 8:19:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 8:19:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 8:19:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 8:19:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 8:19:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 8:19:23 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 20:19:24 dist-test-slave-6k22 krb5kdc[17826](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786306764, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.61.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:19:24.617497 5364 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.842s user 0.134s sys 0.422s
I20260809 20:19:24.617882 5364 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 20:19:24.618115 5364 mini_ranger_kms.cc:401] 127.5.61.20:51907/kms/v1/keys
Aug 09 20:19:24 dist-test-slave-6k22 krb5kdc[17826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306764, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.61.20@KRBTEST.COM for HTTP/127.5.61.20@KRBTEST.COM
Aug 09 20:19:25 dist-test-slave-6k22 krb5kdc[17826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306764, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.61.20@KRBTEST.COM for HTTP/127.5.61.20@KRBTEST.COM
Aug 09 20:19:27 dist-test-slave-6k22 krb5kdc[17826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:19:27.446664 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:34737
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:34737
--ranger_config_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.61.20:51907/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:19:27.804965 18319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:19:27.805243 18319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:19:27.805401 18319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:19:27.812268 18319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:19:27.812426 18319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:19:27.812510 18319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:19:27.812590 18319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:19:27.812660 18319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:19:27.822467 18319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.61.20:51907/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:34737
--ranger_config_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:34737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:19:27.823736 18319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:19:27.825037 18319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:19:27.837146 18324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:19:27.837060 18325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:19:27.837203 18327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:19:27.838040 18319 server_base.cc:1034] running on GCE node
I20260809 20:19:27.838804 18319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:19:27.840194 18319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:19:27.841416 18319 hybrid_clock.cc:648] HybridClock initialized: now 1786306767841376 us; error 53 us; skew 500 ppm
I20260809 20:19:27.841775 18319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:19:27 dist-test-slave-6k22 krb5kdc[17826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306767, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:19:27.849421 18319 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:19:27.850852 18319 webserver.cc:460] Webserver started at http://127.5.61.62:34471/ using document root <none> and password file <none>
I20260809 20:19:27.851465 18319 fs_manager.cc:359] Metadata directory not provided
I20260809 20:19:27.851627 18319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:19:27.852046 18319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 20:19:27 dist-test-slave-6k22 krb5kdc[17826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306767, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for HTTP/127.5.61.20@KRBTEST.COM
I20260809 20:19:27.899989 18319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "95c688896e7547ddbd98661b1d7b7270"
format_stamp: "Formatted at 2026-08-09 20:19:27 on dist-test-slave-6k22"
server_key: "264d91437726f4919cc502fb03f43a0c"
server_key_iv: "edee4b132dfe89ee21b35fbd04fb61fe"
server_key_version: "kuduclusterkey@0"
I20260809 20:19:27.900808 18319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "95c688896e7547ddbd98661b1d7b7270"
format_stamp: "Formatted at 2026-08-09 20:19:27 on dist-test-slave-6k22"
server_key: "264d91437726f4919cc502fb03f43a0c"
server_key_iv: "edee4b132dfe89ee21b35fbd04fb61fe"
server_key_version: "kuduclusterkey@0"
I20260809 20:19:27.906622 18319 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 20:19:27.922583 18334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:19:27.924361 18319 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:19:27.924544 18319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "95c688896e7547ddbd98661b1d7b7270"
format_stamp: "Formatted at 2026-08-09 20:19:27 on dist-test-slave-6k22"
server_key: "264d91437726f4919cc502fb03f43a0c"
server_key_iv: "edee4b132dfe89ee21b35fbd04fb61fe"
server_key_version: "kuduclusterkey@0"
I20260809 20:19:27.924782 18319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:19:27.941874 18319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:19:27.945663 18319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:19:27.945941 18319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:19:27.966265 18319 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:34737
I20260809 20:19:27.966269 18392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:34737 every 8 connection(s)
I20260809 20:19:27.967650 18319 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:19:27.968930 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 18319
I20260809 20:19:27.969174 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:19:27.970160 5364 external_mini_cluster.cc:1428] Setting key 0c67bb695d0cdebbb6ef28d129de1026
I20260809 20:19:27.972306 18393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:19:27.987074 18393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95c688896e7547ddbd98661b1d7b7270: Bootstrap starting.
I20260809 20:19:27.991088 18393 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95c688896e7547ddbd98661b1d7b7270: Neither blocks nor log segments found. Creating new log.
I20260809 20:19:27.992273 18393 log.cc:824] T 00000000000000000000000000000000 P 95c688896e7547ddbd98661b1d7b7270: Log is configured to *not* fsync() on all Append() calls
Aug 09 20:19:27 dist-test-slave-6k22 krb5kdc[17826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:19:27.994941 18393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95c688896e7547ddbd98661b1d7b7270: No bootstrap required, opened a new log
I20260809 20:19:28.003288 18396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:19:27.972838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38824 (local address 127.5.61.62:34737)
0809 20:19:27.984653 (+ 11815us) server_negotiation.cc:206] Beginning negotiation
0809 20:19:27.984669 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 20:19:27.984735 (+ 66us) server_negotiation.cc:407] Connection header received
0809 20:19:27.986335 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:19:27.986354 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:19:27.986707 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:19:27.987064 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:19:27.988379 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:27.989589 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:27.990717 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:27.991420 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:27.991638 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:19:27.995808 (+ 4170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:19:27.995816 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:19:27.995830 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:19:27.995863 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:19:27.998914 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:19:27.999613 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:19:27.999619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:19:27.999656 (+ 37us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:19:27.999819 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:19:28.000226 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:19:28.000241 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:19:28.000243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:19:28.000752 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:19:28.001060 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 20:19:28.001518 (+ 458us) server_negotiation.cc:299] Negotiation successful
0809 20:19:28.001886 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11574,"thread_start_us":11340,"threads_started":1}
I20260809 20:19:28.003510 18393 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95c688896e7547ddbd98661b1d7b7270 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c688896e7547ddbd98661b1d7b7270" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 34737 } }
I20260809 20:19:28.004033 18393 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95c688896e7547ddbd98661b1d7b7270 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:19:28.004176 18393 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95c688896e7547ddbd98661b1d7b7270 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95c688896e7547ddbd98661b1d7b7270, State: Initialized, Role: FOLLOWER
I20260809 20:19:28.004807 18393 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95c688896e7547ddbd98661b1d7b7270 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c688896e7547ddbd98661b1d7b7270" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 34737 } }
I20260809 20:19:28.005090 18393 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95c688896e7547ddbd98661b1d7b7270 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:19:28.005267 18393 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95c688896e7547ddbd98661b1d7b7270 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:19:28.005460 18393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95c688896e7547ddbd98661b1d7b7270 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:19:28.008086 18393 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95c688896e7547ddbd98661b1d7b7270 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c688896e7547ddbd98661b1d7b7270" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 34737 } }
I20260809 20:19:28.008597 18393 leader_election.cc:302] T 00000000000000000000000000000000 P 95c688896e7547ddbd98661b1d7b7270 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95c688896e7547ddbd98661b1d7b7270; no voters:
I20260809 20:19:28.009212 18393 leader_election.cc:288] T 00000000000000000000000000000000 P 95c688896e7547ddbd98661b1d7b7270 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:19:28.009330 18398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95c688896e7547ddbd98661b1d7b7270 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:19:28.011299 18398 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95c688896e7547ddbd98661b1d7b7270 [term 1 LEADER]: Becoming Leader. State: Replica: 95c688896e7547ddbd98661b1d7b7270, State: Running, Role: LEADER
I20260809 20:19:28.011649 18393 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95c688896e7547ddbd98661b1d7b7270 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:19:28.012004 18398 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95c688896e7547ddbd98661b1d7b7270 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c688896e7547ddbd98661b1d7b7270" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 34737 } }
I20260809 20:19:28.015426 18399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95c688896e7547ddbd98661b1d7b7270 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95c688896e7547ddbd98661b1d7b7270" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c688896e7547ddbd98661b1d7b7270" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 34737 } } }
I20260809 20:19:28.015293 18402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95c688896e7547ddbd98661b1d7b7270 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95c688896e7547ddbd98661b1d7b7270. Latest consensus state: current_term: 1 leader_uuid: "95c688896e7547ddbd98661b1d7b7270" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c688896e7547ddbd98661b1d7b7270" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 34737 } } }
I20260809 20:19:28.016079 18399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95c688896e7547ddbd98661b1d7b7270 [sys.catalog]: This master's current role is: LEADER
I20260809 20:19:28.016093 18402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95c688896e7547ddbd98661b1d7b7270 [sys.catalog]: This master's current role is: LEADER
I20260809 20:19:28.020100 18411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:19:28.025239 18411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:19:28.030344 18393 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 20:19:28.043177 18411 catalog_manager.cc:1347] Generated new cluster ID: 773c096573f14664ab55ab7ae7c24af2
I20260809 20:19:28.043378 18411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:19:28.110271 18411 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:19:28.112565 18411 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:19:28.125560 18411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95c688896e7547ddbd98661b1d7b7270: Generated new TSK 0
I20260809 20:19:28.126679 18411 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.61.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
I20260809 20:19:28.873068 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.61.20:51907/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:34737
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 20:19:29 dist-test-slave-6k22 krb5kdc[17826](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786306769, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 20:19:29.311777 18445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:19:29.312158 18445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:19:29.312368 18445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:19:29.323964 18445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:19:29.324184 18445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:19:29.324420 18445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:19:29.340035 18445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.61.20:51907/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:34737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:19:29.341683 18445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:19:29.343411 18445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:19:29.362145 18456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:19:29.368027 18461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:19:29.368213 18459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:19:29.368717 18445 server_base.cc:1034] running on GCE node
I20260809 20:19:29.369474 18445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:19:29.371048 18445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:19:29.372450 18445 hybrid_clock.cc:648] HybridClock initialized: now 1786306769372315 us; error 99 us; skew 500 ppm
I20260809 20:19:29.372949 18445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:19:29 dist-test-slave-6k22 krb5kdc[17826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306769, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:19:29.378561 18445 init.cc:377] Logged in from keytab as kudu/127.5.61.1@KRBTEST.COM (short username kudu)
I20260809 20:19:29.403327 18445 webserver.cc:460] Webserver started at http://127.5.61.1:42895/ using document root <none> and password file <none>
I20260809 20:19:29.404522 18445 fs_manager.cc:359] Metadata directory not provided
I20260809 20:19:29.404767 18445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:19:29.405217 18445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 20:19:29 dist-test-slave-6k22 krb5kdc[17826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306769, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for HTTP/127.5.61.20@KRBTEST.COM
I20260809 20:19:29.426095 18445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "c2057183ee0a4a74bda5cdbc575d3ca0"
format_stamp: "Formatted at 2026-08-09 20:19:29 on dist-test-slave-6k22"
server_key: "5269bd362bb228e8e9664c5e6772916a"
server_key_iv: "4201020ea54e45b1d72c31c04b79bde4"
server_key_version: "kuduclusterkey@0"
I20260809 20:19:29.427109 18445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "c2057183ee0a4a74bda5cdbc575d3ca0"
format_stamp: "Formatted at 2026-08-09 20:19:29 on dist-test-slave-6k22"
server_key: "5269bd362bb228e8e9664c5e6772916a"
server_key_iv: "4201020ea54e45b1d72c31c04b79bde4"
server_key_version: "kuduclusterkey@0"
I20260809 20:19:29.433590 18445 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.008s
I20260809 20:19:29.463726 18470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:19:29.473242 18445 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.001s sys 0.001s
I20260809 20:19:29.473508 18445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "c2057183ee0a4a74bda5cdbc575d3ca0"
format_stamp: "Formatted at 2026-08-09 20:19:29 on dist-test-slave-6k22"
server_key: "5269bd362bb228e8e9664c5e6772916a"
server_key_iv: "4201020ea54e45b1d72c31c04b79bde4"
server_key_version: "kuduclusterkey@0"
I20260809 20:19:29.473789 18445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:19:29.524989 18445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:19:29.530774 18445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:19:29.531196 18445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:19:29.553076 18445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:19:29.561506 18445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:19:29.561712 18445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:19:29.561946 18445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:19:29.562083 18445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 20:19:29 dist-test-slave-6k22 krb5kdc[17826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306769, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for HTTP/127.5.61.20@KRBTEST.COM
I20260809 20:19:29.957401 18445 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:37073
I20260809 20:19:29.957463 18591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:37073 every 8 connection(s)
I20260809 20:19:29.958879 18445 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:19:29.966497 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 18445
I20260809 20:19:29.966727 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:19:29.967660 5364 external_mini_cluster.cc:1428] Setting key 7843971c019802c2c34c66744d58bb40
Aug 09 20:19:29 dist-test-slave-6k22 krb5kdc[17826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306769, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:19:29.989530 18594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:19:29.962617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:39247 (local address 127.5.61.62:34737)
0809 20:19:29.963314 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0809 20:19:29.963320 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:19:29.964253 (+ 933us) server_negotiation.cc:407] Connection header received
0809 20:19:29.969016 (+ 4763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:19:29.969022 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:19:29.969101 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:19:29.969224 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:19:29.971321 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:29.972491 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:29.974813 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:29.975437 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:29.975526 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:19:29.980771 (+ 5245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:19:29.980781 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:19:29.980790 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:19:29.980859 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:19:29.984949 (+ 4090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:19:29.986040 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:19:29.986049 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:19:29.986055 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:19:29.986242 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:19:29.986877 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:19:29.986885 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:19:29.986892 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:19:29.987321 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:19:29.987461 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 20:19:29.988638 (+ 1177us) server_negotiation.cc:299] Negotiation successful
0809 20:19:29.988914 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":419,"threads_started":1}
I20260809 20:19:29.992458 18595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:19:29.963286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:34737 (local address 127.5.61.1:39247)
0809 20:19:29.964068 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0809 20:19:29.964120 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 20:19:29.968699 (+ 4579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:19:29.969783 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:19:29.969795 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:19:29.970380 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:19:29.971125 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:19:29.971149 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:19:29.972705 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:19:29.972710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:19:29.974640 (+ 1930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:19:29.974656 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:19:29.975864 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:19:29.975874 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:19:29.976208 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:19:29.977025 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:19:29.977058 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:19:29.980482 (+ 3424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:19:29.985145 (+ 4663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:19:29.985157 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:19:29.985175 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:19:29.985887 (+ 712us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:19:29.986406 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:19:29.986413 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:19:29.986419 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:19:29.986713 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:19:29.987630 (+ 917us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:19:29.987642 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:19:29.988107 (+ 465us) client_negotiation.cc:769] Sending connection context
0809 20:19:29.990130 (+ 2023us) client_negotiation.cc:240] Negotiation successful
0809 20:19:29.990742 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":327,"threads_started":1}
I20260809 20:19:29.994765 18592 heartbeater.cc:347] Connected to a master server at 127.5.61.62:34737
I20260809 20:19:29.995100 18592 heartbeater.cc:464] Registering TS with master...
I20260809 20:19:29.996068 18592 heartbeater.cc:511] Master 127.5.61.62:34737 requested a full tablet report, sending...
I20260809 20:19:29.998950 18352 ts_manager.cc:194] Registered new tserver with Master: c2057183ee0a4a74bda5cdbc575d3ca0 (127.5.61.1:37073)
I20260809 20:19:30.001235 18352 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.1@KRBTEST.COM'} at 127.5.61.1:39247
WARNING: no policy specified for kudu/127.5.61.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
I20260809 20:19:30.046916 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.61.20:51907/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:34737
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 20:19:30 dist-test-slave-6k22 krb5kdc[17826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306769, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for HTTP/127.5.61.20@KRBTEST.COM
W20260809 20:19:30.536262 18598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:19:30.536778 18598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:19:30.537001 18598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:19:30.548583 18598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:19:30.548916 18598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:19:30.549177 18598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:19:30.562139 18598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.61.20:51907/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:34737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:19:30.563513 18598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:19:30.564931 18598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:19:30.582610 18616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:19:30.584712 18615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:19:30.585541 18619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:19:30.585829 18598 server_base.cc:1034] running on GCE node
I20260809 20:19:30.586454 18598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:19:30.587481 18598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:19:30.588763 18598 hybrid_clock.cc:648] HybridClock initialized: now 1786306770588683 us; error 74 us; skew 500 ppm
I20260809 20:19:30.589092 18598 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:19:30 dist-test-slave-6k22 krb5kdc[17826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306770, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:19:30.592412 18598 init.cc:377] Logged in from keytab as kudu/127.5.61.2@KRBTEST.COM (short username kudu)
I20260809 20:19:30.594267 18598 webserver.cc:460] Webserver started at http://127.5.61.2:32809/ using document root <none> and password file <none>
I20260809 20:19:30.594969 18598 fs_manager.cc:359] Metadata directory not provided
I20260809 20:19:30.595110 18598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:19:30.595448 18598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 20:19:30 dist-test-slave-6k22 krb5kdc[17826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306770, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for HTTP/127.5.61.20@KRBTEST.COM
I20260809 20:19:30.611502 18598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "e80cca80ce134509ada4f2ac53bbfa87"
format_stamp: "Formatted at 2026-08-09 20:19:30 on dist-test-slave-6k22"
server_key: "e70b2f9e9ede13e3a61882d38594a8bd"
server_key_iv: "9e4584aff44a1b825786da3318b6d11d"
server_key_version: "kuduclusterkey@0"
I20260809 20:19:30.612720 18598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "e80cca80ce134509ada4f2ac53bbfa87"
format_stamp: "Formatted at 2026-08-09 20:19:30 on dist-test-slave-6k22"
server_key: "e70b2f9e9ede13e3a61882d38594a8bd"
server_key_iv: "9e4584aff44a1b825786da3318b6d11d"
server_key_version: "kuduclusterkey@0"
I20260809 20:19:30.625383 18598 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.008s sys 0.003s
I20260809 20:19:30.650007 18626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:19:30.651971 18598 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.006s sys 0.004s
I20260809 20:19:30.652246 18598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "e80cca80ce134509ada4f2ac53bbfa87"
format_stamp: "Formatted at 2026-08-09 20:19:30 on dist-test-slave-6k22"
server_key: "e70b2f9e9ede13e3a61882d38594a8bd"
server_key_iv: "9e4584aff44a1b825786da3318b6d11d"
server_key_version: "kuduclusterkey@0"
I20260809 20:19:30.652611 18598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:19:30.697121 18598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:19:30.701365 18598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:19:30.701709 18598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:19:30.702821 18598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:19:30.704376 18598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:19:30.704481 18598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:19:30.704613 18598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:19:30.704689 18598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:19:30.753520 18598 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:45149
I20260809 20:19:30.753548 18744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:45149 every 8 connection(s)
I20260809 20:19:30.754915 18598 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:19:30.762069 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 18598
I20260809 20:19:30.762297 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:19:30.763018 5364 external_mini_cluster.cc:1428] Setting key cd2105b4b4f439c98c32a8f9afbe8297
Aug 09 20:19:30 dist-test-slave-6k22 krb5kdc[17826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306770, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:19:30.791020 18747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:19:30.758682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:57725 (local address 127.5.61.62:34737)
0809 20:19:30.771334 (+ 12652us) server_negotiation.cc:206] Beginning negotiation
0809 20:19:30.771341 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:19:30.771360 (+ 19us) server_negotiation.cc:407] Connection header received
0809 20:19:30.771416 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:19:30.771421 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:19:30.771498 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:19:30.771640 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:19:30.774766 (+ 3126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:30.775827 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:30.778194 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:30.778850 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:30.778952 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:19:30.783932 (+ 4980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:19:30.783939 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:19:30.783945 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:19:30.783993 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:19:30.787437 (+ 3444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:19:30.788333 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:19:30.788338 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:19:30.788342 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:19:30.788490 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:19:30.788987 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:19:30.788992 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:19:30.788996 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:19:30.789358 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:19:30.789486 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 20:19:30.790402 (+ 916us) server_negotiation.cc:299] Negotiation successful
0809 20:19:30.790628 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12516,"thread_start_us":12393,"threads_started":1}
I20260809 20:19:30.793612 18748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:19:30.759089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:34737 (local address 127.5.61.2:57725)
0809 20:19:30.768175 (+ 9086us) negotiation.cc:107] Waiting for socket to connect
0809 20:19:30.768240 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 20:19:30.770570 (+ 2330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:19:30.773102 (+ 2532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:19:30.773118 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:19:30.773790 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:19:30.774587 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:19:30.774614 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:19:30.776009 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:19:30.776014 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:19:30.777997 (+ 1983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:19:30.778023 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:19:30.779297 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:19:30.779305 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:19:30.779613 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:19:30.780492 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:19:30.780521 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:19:30.783658 (+ 3137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:19:30.787626 (+ 3968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:19:30.787634 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:19:30.787650 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:19:30.788197 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:19:30.788629 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:19:30.788634 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:19:30.788637 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:19:30.788864 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:19:30.789580 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:19:30.789588 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:19:30.789970 (+ 382us) client_negotiation.cc:769] Sending connection context
0809 20:19:30.791649 (+ 1679us) client_negotiation.cc:240] Negotiation successful
0809 20:19:30.792185 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8748,"thread_start_us":11989,"threads_started":1}
I20260809 20:19:30.795651 18745 heartbeater.cc:347] Connected to a master server at 127.5.61.62:34737
I20260809 20:19:30.795962 18745 heartbeater.cc:464] Registering TS with master...
I20260809 20:19:30.796864 18745 heartbeater.cc:511] Master 127.5.61.62:34737 requested a full tablet report, sending...
I20260809 20:19:30.798609 18352 ts_manager.cc:194] Registered new tserver with Master: e80cca80ce134509ada4f2ac53bbfa87 (127.5.61.2:45149)
WARNING: no policy specified for kudu/127.5.61.3@KRBTEST.COM; defaulting to no policy
I20260809 20:19:30.799712 18352 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.2@KRBTEST.COM'} at 127.5.61.2:57725
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
I20260809 20:19:30.847314 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.61.20:51907/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:34737
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 20:19:31.005208 18592 heartbeater.cc:503] Master 127.5.61.62:34737 was elected leader, sending a full tablet report...
W20260809 20:19:31.253041 18751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:19:31.253394 18751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:19:31.253566 18751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:19:31.261358 18751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:19:31.261538 18751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:19:31.261722 18751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:19:31.271651 18751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.61.20:51907/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:34737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:19:31.273008 18751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:19:31.274392 18751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:19:31.287997 18756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:19:31.288033 18760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:19:31.287961 18758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:19:31.288990 18751 server_base.cc:1034] running on GCE node
I20260809 20:19:31.289574 18751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:19:31.290522 18751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:19:31.291692 18751 hybrid_clock.cc:648] HybridClock initialized: now 1786306771291602 us; error 51 us; skew 500 ppm
I20260809 20:19:31.292047 18751 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:19:31 dist-test-slave-6k22 krb5kdc[17826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306771, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:19:31.295323 18751 init.cc:377] Logged in from keytab as kudu/127.5.61.3@KRBTEST.COM (short username kudu)
I20260809 20:19:31.296808 18751 webserver.cc:460] Webserver started at http://127.5.61.3:33349/ using document root <none> and password file <none>
I20260809 20:19:31.297505 18751 fs_manager.cc:359] Metadata directory not provided
I20260809 20:19:31.297662 18751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:19:31.298003 18751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 20:19:31 dist-test-slave-6k22 krb5kdc[17826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306771, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for HTTP/127.5.61.20@KRBTEST.COM
I20260809 20:19:31.314930 18751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "882a5ae67bb543619a2017082aa2522b"
format_stamp: "Formatted at 2026-08-09 20:19:31 on dist-test-slave-6k22"
server_key: "ecd6363ac02b7b3706775cafa52f8140"
server_key_iv: "cddad4685eda295a2bb32cec626107dc"
server_key_version: "kuduclusterkey@0"
I20260809 20:19:31.316113 18751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "882a5ae67bb543619a2017082aa2522b"
format_stamp: "Formatted at 2026-08-09 20:19:31 on dist-test-slave-6k22"
server_key: "ecd6363ac02b7b3706775cafa52f8140"
server_key_iv: "cddad4685eda295a2bb32cec626107dc"
server_key_version: "kuduclusterkey@0"
I20260809 20:19:31.323098 18751 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.005s
I20260809 20:19:31.338054 18768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:19:31.339825 18751 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 20:19:31.340001 18751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "882a5ae67bb543619a2017082aa2522b"
format_stamp: "Formatted at 2026-08-09 20:19:31 on dist-test-slave-6k22"
server_key: "ecd6363ac02b7b3706775cafa52f8140"
server_key_iv: "cddad4685eda295a2bb32cec626107dc"
server_key_version: "kuduclusterkey@0"
I20260809 20:19:31.340220 18751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:19:31.370419 18751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:19:31.374464 18751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:19:31.374794 18751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:19:31.375830 18751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:19:31.377131 18751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:19:31.377264 18751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:19:31.377413 18751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:19:31.377489 18751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:19:31.411994 18751 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:46531
I20260809 20:19:31.412111 18895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:46531 every 8 connection(s)
I20260809 20:19:31.413478 18751 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:19:31.420934 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 18751
I20260809 20:19:31.421082 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:19:31.421494 5364 external_mini_cluster.cc:1428] Setting key c6fc1c10ea01511d2c5d76858f05ab6a
Aug 09 20:19:31 dist-test-slave-6k22 krb5kdc[17826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306771, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:19:31.433313 18898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:19:31.416042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:56777 (local address 127.5.61.62:34737)
0809 20:19:31.416542 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0809 20:19:31.416546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:19:31.417305 (+ 759us) server_negotiation.cc:407] Connection header received
0809 20:19:31.419152 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:19:31.419157 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:19:31.419257 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:19:31.419387 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:19:31.420955 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.421834 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.423534 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.424009 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.424105 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:19:31.427289 (+ 3184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:19:31.427293 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:19:31.427297 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:19:31.427331 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:19:31.430118 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:19:31.430812 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:19:31.430825 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:19:31.430829 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:19:31.430992 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:19:31.431462 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:19:31.431473 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:19:31.431476 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:19:31.431828 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:19:31.431991 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 20:19:31.432663 (+ 672us) server_negotiation.cc:299] Negotiation successful
0809 20:19:31.432908 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":260,"threads_started":1}
I20260809 20:19:31.434237 18899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:19:31.416476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:34737 (local address 127.5.61.3:56777)
0809 20:19:31.417130 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0809 20:19:31.417173 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 20:19:31.418894 (+ 1721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:19:31.419697 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:19:31.419709 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:19:31.420189 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:19:31.420784 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:19:31.420802 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.422000 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.422005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:19:31.423352 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:19:31.423381 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.424232 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.424242 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:19:31.424446 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:19:31.425107 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:19:31.425129 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:19:31.427075 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:19:31.430264 (+ 3189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:19:31.430273 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:19:31.430294 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:19:31.430692 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:19:31.431104 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:19:31.431109 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:19:31.431113 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:19:31.431310 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:19:31.431969 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:19:31.431977 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:19:31.432287 (+ 310us) client_negotiation.cc:769] Sending connection context
0809 20:19:31.432559 (+ 272us) client_negotiation.cc:240] Negotiation successful
0809 20:19:31.433136 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":269,"threads_started":1}
I20260809 20:19:31.435781 18896 heartbeater.cc:347] Connected to a master server at 127.5.61.62:34737
I20260809 20:19:31.436110 18896 heartbeater.cc:464] Registering TS with master...
I20260809 20:19:31.436932 18896 heartbeater.cc:511] Master 127.5.61.62:34737 requested a full tablet report, sending...
I20260809 20:19:31.438419 18352 ts_manager.cc:194] Registered new tserver with Master: 882a5ae67bb543619a2017082aa2522b (127.5.61.3:46531)
I20260809 20:19:31.439216 18352 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.3@KRBTEST.COM'} at 127.5.61.3:56777
I20260809 20:19:31.448942 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 20:19:31.466212 18898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:19:31.453383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36172 (local address 127.5.61.62:34737)
0809 20:19:31.453595 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 20:19:31.453603 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:19:31.454007 (+ 404us) server_negotiation.cc:407] Connection header received
0809 20:19:31.454217 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:19:31.454222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:19:31.454304 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:19:31.454406 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:19:31.455884 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.456613 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.457961 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.458443 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.458568 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:19:31.461169 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:19:31.461173 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:19:31.461176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:19:31.461204 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:19:31.463975 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:19:31.464601 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:19:31.464605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:19:31.464607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:19:31.464720 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:19:31.465036 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:19:31.465039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:19:31.465041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:19:31.465296 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:19:31.465489 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 20:19:31.465711 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 20:19:31.465870 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 20:19:31.470050 18352 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36172:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:19:31.477137 18352 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:19:31.514209 18912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:19:31.490540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:46531 (local address 127.0.0.1:58004)
0809 20:19:31.506271 (+ 15731us) negotiation.cc:107] Waiting for socket to connect
0809 20:19:31.506288 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:19:31.506440 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:19:31.507210 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:19:31.507214 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:19:31.507246 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:19:31.507684 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:19:31.507700 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.509545 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.509550 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:19:31.511258 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:19:31.511270 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.513239 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.513246 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:19:31.513429 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:19:31.513486 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 20:19:31.513627 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 20:19:31.513846 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15600,"thread_start_us":14297,"threads_started":1}
I20260809 20:19:31.514238 18914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:19:31.490620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58004 (local address 127.5.61.3:46531)
0809 20:19:31.504402 (+ 13782us) server_negotiation.cc:206] Beginning negotiation
0809 20:19:31.504414 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 20:19:31.506412 (+ 1998us) server_negotiation.cc:407] Connection header received
0809 20:19:31.506634 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:19:31.506641 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:19:31.506858 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:19:31.507045 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:19:31.507924 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.509411 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.511419 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.512688 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.512856 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:19:31.513093 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0809 20:19:31.513692 (+ 599us) server_negotiation.cc:299] Negotiation successful
0809 20:19:31.513887 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13569,"thread_start_us":13801,"threads_started":1}
I20260809 20:19:31.515080 18913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:19:31.489813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:45149 (local address 127.0.0.1:50498)
0809 20:19:31.505141 (+ 15328us) negotiation.cc:107] Waiting for socket to connect
0809 20:19:31.505167 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 20:19:31.505328 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:19:31.506526 (+ 1198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:19:31.506547 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:19:31.506568 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:19:31.507264 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:19:31.507278 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.509120 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.509128 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:19:31.512185 (+ 3057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:19:31.512194 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.514196 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.514203 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:19:31.514416 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:19:31.514448 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 20:19:31.514533 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 20:19:31.514652 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15176,"mutex_wait_us":417,"thread_start_us":16135,"threads_started":1}
I20260809 20:19:31.515175 18915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:19:31.490970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50498 (local address 127.5.61.2:45149)
0809 20:19:31.505826 (+ 14856us) server_negotiation.cc:206] Beginning negotiation
0809 20:19:31.505837 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:19:31.505882 (+ 45us) server_negotiation.cc:407] Connection header received
0809 20:19:31.506012 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:19:31.506021 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:19:31.506316 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:19:31.506504 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:19:31.507448 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.508986 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.512343 (+ 3357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.513753 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.513864 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:19:31.514046 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 20:19:31.514613 (+ 567us) server_negotiation.cc:299] Negotiation successful
0809 20:19:31.514785 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14711,"thread_start_us":14600,"threads_started":1}
I20260809 20:19:31.515877 18910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:19:31.489391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37073 (local address 127.0.0.1:44218)
0809 20:19:31.496476 (+ 7085us) negotiation.cc:107] Waiting for socket to connect
0809 20:19:31.496537 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 20:19:31.496766 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:19:31.506185 (+ 9419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:19:31.506192 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:19:31.506243 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:19:31.506949 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:19:31.506963 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.509240 (+ 2277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.509244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:19:31.512165 (+ 2921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:19:31.512186 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.514451 (+ 2265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.514457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:19:31.514665 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:19:31.514697 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 20:19:31.514791 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 20:19:31.514957 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6751,"mutex_wait_us":167,"thread_start_us":6575,"threads_started":1}
I20260809 20:19:31.516324 18911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:19:31.489597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44218 (local address 127.5.61.1:37073)
0809 20:19:31.505070 (+ 15473us) server_negotiation.cc:206] Beginning negotiation
0809 20:19:31.505083 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 20:19:31.505121 (+ 38us) server_negotiation.cc:407] Connection header received
0809 20:19:31.505261 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:19:31.505272 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:19:31.505605 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:19:31.506737 (+ 1132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:19:31.507166 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.509046 (+ 1880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.512380 (+ 3334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.513918 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.514053 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:19:31.514276 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 20:19:31.515177 (+ 901us) server_negotiation.cc:299] Negotiation successful
0809 20:19:31.515370 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15222,"thread_start_us":16207,"threads_started":1}
I20260809 20:19:31.518767 18678 tablet_service.cc:1467] Processing CreateTablet for tablet d9d7f861ec9e4be0b6910d9ca02af7fc (DEFAULT_TABLE table=test-table [id=5cb54dea633e4b118bf6aef2e8a5c024]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:19:31.518818 18822 tablet_service.cc:1467] Processing CreateTablet for tablet d9d7f861ec9e4be0b6910d9ca02af7fc (DEFAULT_TABLE table=test-table [id=5cb54dea633e4b118bf6aef2e8a5c024]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:19:31.520161 18678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9d7f861ec9e4be0b6910d9ca02af7fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:19:31.520339 18822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9d7f861ec9e4be0b6910d9ca02af7fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:19:31.519990 18523 tablet_service.cc:1467] Processing CreateTablet for tablet d9d7f861ec9e4be0b6910d9ca02af7fc (DEFAULT_TABLE table=test-table [id=5cb54dea633e4b118bf6aef2e8a5c024]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:19:31.521270 18523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9d7f861ec9e4be0b6910d9ca02af7fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:19:31.531706 18916 tablet_bootstrap.cc:492] T d9d7f861ec9e4be0b6910d9ca02af7fc P e80cca80ce134509ada4f2ac53bbfa87: Bootstrap starting.
I20260809 20:19:31.534487 18917 tablet_bootstrap.cc:492] T d9d7f861ec9e4be0b6910d9ca02af7fc P 882a5ae67bb543619a2017082aa2522b: Bootstrap starting.
I20260809 20:19:31.535403 18918 tablet_bootstrap.cc:492] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0: Bootstrap starting.
I20260809 20:19:31.536499 18916 tablet_bootstrap.cc:654] T d9d7f861ec9e4be0b6910d9ca02af7fc P e80cca80ce134509ada4f2ac53bbfa87: Neither blocks nor log segments found. Creating new log.
I20260809 20:19:31.537753 18916 log.cc:824] T d9d7f861ec9e4be0b6910d9ca02af7fc P e80cca80ce134509ada4f2ac53bbfa87: Log is configured to *not* fsync() on all Append() calls
I20260809 20:19:31.538210 18917 tablet_bootstrap.cc:654] T d9d7f861ec9e4be0b6910d9ca02af7fc P 882a5ae67bb543619a2017082aa2522b: Neither blocks nor log segments found. Creating new log.
I20260809 20:19:31.538525 18918 tablet_bootstrap.cc:654] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0: Neither blocks nor log segments found. Creating new log.
I20260809 20:19:31.539512 18917 log.cc:824] T d9d7f861ec9e4be0b6910d9ca02af7fc P 882a5ae67bb543619a2017082aa2522b: Log is configured to *not* fsync() on all Append() calls
I20260809 20:19:31.539846 18918 log.cc:824] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0: Log is configured to *not* fsync() on all Append() calls
I20260809 20:19:31.541275 18916 tablet_bootstrap.cc:492] T d9d7f861ec9e4be0b6910d9ca02af7fc P e80cca80ce134509ada4f2ac53bbfa87: No bootstrap required, opened a new log
I20260809 20:19:31.541559 18916 ts_tablet_manager.cc:1397] T d9d7f861ec9e4be0b6910d9ca02af7fc P e80cca80ce134509ada4f2ac53bbfa87: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260809 20:19:31.542500 18917 tablet_bootstrap.cc:492] T d9d7f861ec9e4be0b6910d9ca02af7fc P 882a5ae67bb543619a2017082aa2522b: No bootstrap required, opened a new log
I20260809 20:19:31.542776 18917 ts_tablet_manager.cc:1397] T d9d7f861ec9e4be0b6910d9ca02af7fc P 882a5ae67bb543619a2017082aa2522b: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 20:19:31.542831 18918 tablet_bootstrap.cc:492] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0: No bootstrap required, opened a new log
I20260809 20:19:31.543066 18918 ts_tablet_manager.cc:1397] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260809 20:19:31.550058 18916 raft_consensus.cc:348] T d9d7f861ec9e4be0b6910d9ca02af7fc P e80cca80ce134509ada4f2ac53bbfa87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2057183ee0a4a74bda5cdbc575d3ca0" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37073 } } peers { permanent_uuid: "e80cca80ce134509ada4f2ac53bbfa87" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 45149 } } peers { permanent_uuid: "882a5ae67bb543619a2017082aa2522b" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46531 } }
I20260809 20:19:31.550721 18916 raft_consensus.cc:374] T d9d7f861ec9e4be0b6910d9ca02af7fc P e80cca80ce134509ada4f2ac53bbfa87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:19:31.550837 18916 raft_consensus.cc:729] T d9d7f861ec9e4be0b6910d9ca02af7fc P e80cca80ce134509ada4f2ac53bbfa87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e80cca80ce134509ada4f2ac53bbfa87, State: Initialized, Role: FOLLOWER
I20260809 20:19:31.551597 18916 consensus_queue.cc:260] T d9d7f861ec9e4be0b6910d9ca02af7fc P e80cca80ce134509ada4f2ac53bbfa87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2057183ee0a4a74bda5cdbc575d3ca0" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37073 } } peers { permanent_uuid: "e80cca80ce134509ada4f2ac53bbfa87" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 45149 } } peers { permanent_uuid: "882a5ae67bb543619a2017082aa2522b" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46531 } }
I20260809 20:19:31.552804 18745 heartbeater.cc:503] Master 127.5.61.62:34737 was elected leader, sending a full tablet report...
I20260809 20:19:31.553469 18916 ts_tablet_manager.cc:1428] T d9d7f861ec9e4be0b6910d9ca02af7fc P e80cca80ce134509ada4f2ac53bbfa87: Time spent starting tablet: real 0.012s user 0.006s sys 0.006s
I20260809 20:19:31.554504 18917 raft_consensus.cc:348] T d9d7f861ec9e4be0b6910d9ca02af7fc P 882a5ae67bb543619a2017082aa2522b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2057183ee0a4a74bda5cdbc575d3ca0" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37073 } } peers { permanent_uuid: "e80cca80ce134509ada4f2ac53bbfa87" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 45149 } } peers { permanent_uuid: "882a5ae67bb543619a2017082aa2522b" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46531 } }
I20260809 20:19:31.554908 18917 raft_consensus.cc:374] T d9d7f861ec9e4be0b6910d9ca02af7fc P 882a5ae67bb543619a2017082aa2522b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:19:31.555013 18917 raft_consensus.cc:729] T d9d7f861ec9e4be0b6910d9ca02af7fc P 882a5ae67bb543619a2017082aa2522b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 882a5ae67bb543619a2017082aa2522b, State: Initialized, Role: FOLLOWER
I20260809 20:19:31.555178 18918 raft_consensus.cc:348] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2057183ee0a4a74bda5cdbc575d3ca0" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37073 } } peers { permanent_uuid: "e80cca80ce134509ada4f2ac53bbfa87" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 45149 } } peers { permanent_uuid: "882a5ae67bb543619a2017082aa2522b" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46531 } }
I20260809 20:19:31.555586 18918 raft_consensus.cc:374] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:19:31.555743 18918 raft_consensus.cc:729] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2057183ee0a4a74bda5cdbc575d3ca0, State: Initialized, Role: FOLLOWER
I20260809 20:19:31.555632 18917 consensus_queue.cc:260] T d9d7f861ec9e4be0b6910d9ca02af7fc P 882a5ae67bb543619a2017082aa2522b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2057183ee0a4a74bda5cdbc575d3ca0" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37073 } } peers { permanent_uuid: "e80cca80ce134509ada4f2ac53bbfa87" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 45149 } } peers { permanent_uuid: "882a5ae67bb543619a2017082aa2522b" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46531 } }
I20260809 20:19:31.556340 18918 consensus_queue.cc:260] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2057183ee0a4a74bda5cdbc575d3ca0" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37073 } } peers { permanent_uuid: "e80cca80ce134509ada4f2ac53bbfa87" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 45149 } } peers { permanent_uuid: "882a5ae67bb543619a2017082aa2522b" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46531 } }
I20260809 20:19:31.556840 18896 heartbeater.cc:503] Master 127.5.61.62:34737 was elected leader, sending a full tablet report...
I20260809 20:19:31.557546 18917 ts_tablet_manager.cc:1428] T d9d7f861ec9e4be0b6910d9ca02af7fc P 882a5ae67bb543619a2017082aa2522b: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260809 20:19:31.560760 18918 ts_tablet_manager.cc:1428] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0: Time spent starting tablet: real 0.018s user 0.011s sys 0.007s
I20260809 20:19:31.612816 18924 raft_consensus.cc:482] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:19:31.613175 18924 raft_consensus.cc:504] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2057183ee0a4a74bda5cdbc575d3ca0" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37073 } } peers { permanent_uuid: "e80cca80ce134509ada4f2ac53bbfa87" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 45149 } } peers { permanent_uuid: "882a5ae67bb543619a2017082aa2522b" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46531 } }
I20260809 20:19:31.615159 18924 leader_election.cc:288] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e80cca80ce134509ada4f2ac53bbfa87 (127.5.61.2:45149), 882a5ae67bb543619a2017082aa2522b (127.5.61.3:46531)
I20260809 20:19:31.623183 18925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:19:31.615642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:45149 (local address 127.5.61.1:57313)
0809 20:19:31.616662 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0809 20:19:31.616678 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:19:31.616798 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:19:31.617177 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:19:31.617182 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:19:31.617226 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:19:31.618178 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:19:31.618192 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.619747 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.619752 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:19:31.621098 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:19:31.621110 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.622387 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.622393 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:19:31.622566 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:19:31.622609 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 20:19:31.622725 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 20:19:31.622886 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":917,"mutex_wait_us":100,"spinlock_wait_cycles":72960,"thread_start_us":482,"threads_started":1}
I20260809 20:19:31.623243 18915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:19:31.615778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:57313 (local address 127.5.61.2:45149)
0809 20:19:31.616002 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 20:19:31.616010 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:19:31.616780 (+ 770us) server_negotiation.cc:407] Connection header received
0809 20:19:31.616950 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:19:31.616956 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:19:31.617049 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:19:31.617148 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:19:31.618366 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.619508 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.621234 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.622126 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.622251 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:19:31.622318 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 20:19:31.622744 (+ 426us) server_negotiation.cc:299] Negotiation successful
0809 20:19:31.622886 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260809 20:19:31.624482 18698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9d7f861ec9e4be0b6910d9ca02af7fc" candidate_uuid: "c2057183ee0a4a74bda5cdbc575d3ca0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e80cca80ce134509ada4f2ac53bbfa87" is_pre_election: true
I20260809 20:19:31.625079 18914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:19:31.615691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:39369 (local address 127.5.61.3:46531)
0809 20:19:31.615929 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 20:19:31.615936 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:19:31.616780 (+ 844us) server_negotiation.cc:407] Connection header received
0809 20:19:31.616950 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:19:31.616956 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:19:31.617042 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:19:31.617148 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:19:31.618454 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.619506 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.622866 (+ 3360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.624213 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.624311 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:19:31.624380 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 20:19:31.624795 (+ 415us) server_negotiation.cc:299] Negotiation successful
0809 20:19:31.624918 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 20:19:31.625047 18698 raft_consensus.cc:2454] T d9d7f861ec9e4be0b6910d9ca02af7fc P e80cca80ce134509ada4f2ac53bbfa87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2057183ee0a4a74bda5cdbc575d3ca0 in term 0.
I20260809 20:19:31.625253 18926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:19:31.615633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:46531 (local address 127.5.61.1:39369)
0809 20:19:31.616604 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0809 20:19:31.616630 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 20:19:31.616804 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:19:31.617177 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:19:31.617182 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:19:31.617224 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:19:31.618177 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:19:31.618192 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.621244 (+ 3052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.621247 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:19:31.622708 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:19:31.622722 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.624426 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.624432 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:19:31.624603 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:19:31.624643 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 20:19:31.624749 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 20:19:31.624877 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"mutex_wait_us":83,"spinlock_wait_cycles":63616,"thread_start_us":558,"threads_started":1}
I20260809 20:19:31.625818 18472 leader_election.cc:302] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c2057183ee0a4a74bda5cdbc575d3ca0, e80cca80ce134509ada4f2ac53bbfa87; no voters:
I20260809 20:19:31.626312 18924 raft_consensus.cc:2781] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:19:31.626195 18844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9d7f861ec9e4be0b6910d9ca02af7fc" candidate_uuid: "c2057183ee0a4a74bda5cdbc575d3ca0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "882a5ae67bb543619a2017082aa2522b" is_pre_election: true
I20260809 20:19:31.626499 18924 raft_consensus.cc:482] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:19:31.626605 18924 raft_consensus.cc:3037] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:19:31.626804 18844 raft_consensus.cc:2454] T d9d7f861ec9e4be0b6910d9ca02af7fc P 882a5ae67bb543619a2017082aa2522b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2057183ee0a4a74bda5cdbc575d3ca0 in term 0.
I20260809 20:19:31.629102 18924 raft_consensus.cc:504] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2057183ee0a4a74bda5cdbc575d3ca0" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37073 } } peers { permanent_uuid: "e80cca80ce134509ada4f2ac53bbfa87" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 45149 } } peers { permanent_uuid: "882a5ae67bb543619a2017082aa2522b" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46531 } }
I20260809 20:19:31.629805 18924 leader_election.cc:288] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0 [CANDIDATE]: Term 1 election: Requested vote from peers e80cca80ce134509ada4f2ac53bbfa87 (127.5.61.2:45149), 882a5ae67bb543619a2017082aa2522b (127.5.61.3:46531)
I20260809 20:19:31.630188 18698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9d7f861ec9e4be0b6910d9ca02af7fc" candidate_uuid: "c2057183ee0a4a74bda5cdbc575d3ca0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e80cca80ce134509ada4f2ac53bbfa87"
I20260809 20:19:31.630304 18844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9d7f861ec9e4be0b6910d9ca02af7fc" candidate_uuid: "c2057183ee0a4a74bda5cdbc575d3ca0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "882a5ae67bb543619a2017082aa2522b"
I20260809 20:19:31.630409 18698 raft_consensus.cc:3037] T d9d7f861ec9e4be0b6910d9ca02af7fc P e80cca80ce134509ada4f2ac53bbfa87 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:19:31.630513 18844 raft_consensus.cc:3037] T d9d7f861ec9e4be0b6910d9ca02af7fc P 882a5ae67bb543619a2017082aa2522b [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:19:31.633782 18844 raft_consensus.cc:2454] T d9d7f861ec9e4be0b6910d9ca02af7fc P 882a5ae67bb543619a2017082aa2522b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2057183ee0a4a74bda5cdbc575d3ca0 in term 1.
I20260809 20:19:31.633782 18698 raft_consensus.cc:2454] T d9d7f861ec9e4be0b6910d9ca02af7fc P e80cca80ce134509ada4f2ac53bbfa87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2057183ee0a4a74bda5cdbc575d3ca0 in term 1.
I20260809 20:19:31.634173 18472 leader_election.cc:302] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c2057183ee0a4a74bda5cdbc575d3ca0, e80cca80ce134509ada4f2ac53bbfa87; no voters:
I20260809 20:19:31.634452 18924 raft_consensus.cc:2781] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:19:31.634892 18924 raft_consensus.cc:686] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0 [term 1 LEADER]: Becoming Leader. State: Replica: c2057183ee0a4a74bda5cdbc575d3ca0, State: Running, Role: LEADER
I20260809 20:19:31.635434 18924 consensus_queue.cc:237] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2057183ee0a4a74bda5cdbc575d3ca0" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37073 } } peers { permanent_uuid: "e80cca80ce134509ada4f2ac53bbfa87" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 45149 } } peers { permanent_uuid: "882a5ae67bb543619a2017082aa2522b" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46531 } }
I20260809 20:19:31.640594 18350 catalog_manager.cc:5495] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0 reported cstate change: term changed from 0 to 1, leader changed from <none> to c2057183ee0a4a74bda5cdbc575d3ca0 (127.5.61.1). New cstate: current_term: 1 leader_uuid: "c2057183ee0a4a74bda5cdbc575d3ca0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2057183ee0a4a74bda5cdbc575d3ca0" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37073 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e80cca80ce134509ada4f2ac53bbfa87" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 45149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "882a5ae67bb543619a2017082aa2522b" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46531 } health_report { overall_health: UNKNOWN } } }
I20260809 20:19:31.661415 18911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:19:31.655984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44224 (local address 127.5.61.1:37073)
0809 20:19:31.656170 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 20:19:31.656175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:19:31.656195 (+ 20us) server_negotiation.cc:407] Connection header received
0809 20:19:31.656339 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:19:31.656343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:19:31.656429 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:19:31.656569 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:19:31.656911 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.657668 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.659308 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.659781 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.659878 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:19:31.660453 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:19:31.660966 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:19:31.661076 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 20:19:31.661160 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 20:19:31.661222 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 20:19:31.665292 18897 tablet.cc:2369] T d9d7f861ec9e4be0b6910d9ca02af7fc P 882a5ae67bb543619a2017082aa2522b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:19:31.669961 18698 raft_consensus.cc:1261] T d9d7f861ec9e4be0b6910d9ca02af7fc P e80cca80ce134509ada4f2ac53bbfa87 [term 1 FOLLOWER]: Refusing update from remote peer c2057183ee0a4a74bda5cdbc575d3ca0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:19:31.669960 18844 raft_consensus.cc:1261] T d9d7f861ec9e4be0b6910d9ca02af7fc P 882a5ae67bb543619a2017082aa2522b [term 1 FOLLOWER]: Refusing update from remote peer c2057183ee0a4a74bda5cdbc575d3ca0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:19:31.671029 18928 consensus_queue.cc:1035] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "882a5ae67bb543619a2017082aa2522b" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:19:31.671443 18924 consensus_queue.cc:1035] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e80cca80ce134509ada4f2ac53bbfa87" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 45149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:19:31.684844 18933 mvcc.cc:205] Tried to move back new op lower bound from 7316712536748990464 to 7316712536622698496. Current Snapshot: MvccSnapshot[applied={T|T < 7316712536748990464}]
I20260809 20:19:31.684880 18931 mvcc.cc:205] Tried to move back new op lower bound from 7316712536748990464 to 7316712536622698496. Current Snapshot: MvccSnapshot[applied={T|T < 7316712536748990464 or (T in {7316712536748990464})}]
I20260809 20:19:31.696771 18350 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:19:31.697079 18350 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:19:31.700986 18350 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 95c688896e7547ddbd98661b1d7b7270: Sending DeleteTablet for 3 replicas of tablet d9d7f861ec9e4be0b6910d9ca02af7fc
I20260809 20:19:31.702070 18523 tablet_service.cc:1514] Processing DeleteTablet for tablet d9d7f861ec9e4be0b6910d9ca02af7fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:19:31 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:44218
I20260809 20:19:31.702204 18822 tablet_service.cc:1514] Processing DeleteTablet for tablet d9d7f861ec9e4be0b6910d9ca02af7fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:19:31 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:58004
I20260809 20:19:31.702895 18678 tablet_service.cc:1514] Processing DeleteTablet for tablet d9d7f861ec9e4be0b6910d9ca02af7fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:19:31 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:50498
I20260809 20:19:31.704761 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 18445
I20260809 20:19:31.713388 18943 tablet_replica.cc:330] T d9d7f861ec9e4be0b6910d9ca02af7fc P e80cca80ce134509ada4f2ac53bbfa87: stopping tablet replica
I20260809 20:19:31.714403 18943 raft_consensus.cc:2229] T d9d7f861ec9e4be0b6910d9ca02af7fc P e80cca80ce134509ada4f2ac53bbfa87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:19:31.715839 18943 raft_consensus.cc:2258] T d9d7f861ec9e4be0b6910d9ca02af7fc P e80cca80ce134509ada4f2ac53bbfa87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:19:31.716653 18942 tablet_replica.cc:330] T d9d7f861ec9e4be0b6910d9ca02af7fc P 882a5ae67bb543619a2017082aa2522b: stopping tablet replica
I20260809 20:19:31.717187 18942 raft_consensus.cc:2229] T d9d7f861ec9e4be0b6910d9ca02af7fc P 882a5ae67bb543619a2017082aa2522b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:19:31.717692 18942 raft_consensus.cc:2258] T d9d7f861ec9e4be0b6910d9ca02af7fc P 882a5ae67bb543619a2017082aa2522b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:19:31.718989 18941 tablet_replica.cc:330] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0: stopping tablet replica
I20260809 20:19:31.719501 18941 raft_consensus.cc:2229] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:19:31.720263 18941 raft_consensus.cc:2258] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:19:31.720649 18942 ts_tablet_manager.cc:1905] T d9d7f861ec9e4be0b6910d9ca02af7fc P 882a5ae67bb543619a2017082aa2522b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:19:31.720940 18943 ts_tablet_manager.cc:1905] T d9d7f861ec9e4be0b6910d9ca02af7fc P e80cca80ce134509ada4f2ac53bbfa87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:19:31.725296 18941 ts_tablet_manager.cc:1905] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:19:31.727829 18942 ts_tablet_manager.cc:1918] T d9d7f861ec9e4be0b6910d9ca02af7fc P 882a5ae67bb543619a2017082aa2522b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:19:31.727854 18943 ts_tablet_manager.cc:1918] T d9d7f861ec9e4be0b6910d9ca02af7fc P e80cca80ce134509ada4f2ac53bbfa87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:19:31.728008 18943 log.cc:1196] T d9d7f861ec9e4be0b6910d9ca02af7fc P e80cca80ce134509ada4f2ac53bbfa87: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/d9d7f861ec9e4be0b6910d9ca02af7fc
I20260809 20:19:31.728078 18942 log.cc:1196] T d9d7f861ec9e4be0b6910d9ca02af7fc P 882a5ae67bb543619a2017082aa2522b: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/d9d7f861ec9e4be0b6910d9ca02af7fc
I20260809 20:19:31.728693 18943 ts_tablet_manager.cc:1939] T d9d7f861ec9e4be0b6910d9ca02af7fc P e80cca80ce134509ada4f2ac53bbfa87: Deleting consensus metadata
I20260809 20:19:31.728714 18942 ts_tablet_manager.cc:1939] T d9d7f861ec9e4be0b6910d9ca02af7fc P 882a5ae67bb543619a2017082aa2522b: Deleting consensus metadata
I20260809 20:19:31.730640 18337 catalog_manager.cc:4841] TS 882a5ae67bb543619a2017082aa2522b (127.5.61.3:46531): tablet d9d7f861ec9e4be0b6910d9ca02af7fc (table test-table [id=5cb54dea633e4b118bf6aef2e8a5c024]) successfully deleted
I20260809 20:19:31.730719 18336 catalog_manager.cc:4841] TS e80cca80ce134509ada4f2ac53bbfa87 (127.5.61.2:45149): tablet d9d7f861ec9e4be0b6910d9ca02af7fc (table test-table [id=5cb54dea633e4b118bf6aef2e8a5c024]) successfully deleted
Aug 09 20:19:31 dist-test-slave-6k22 krb5kdc[17826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.1@KRBTEST.COM
I20260809 20:19:31.736366 18941 ts_tablet_manager.cc:1918] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:19:31.736552 18941 log.cc:1196] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/d9d7f861ec9e4be0b6910d9ca02af7fc
I20260809 20:19:31.737226 18941 ts_tablet_manager.cc:1939] T d9d7f861ec9e4be0b6910d9ca02af7fc P c2057183ee0a4a74bda5cdbc575d3ca0: Deleting consensus metadata
I20260809 20:19:31.739207 18335 catalog_manager.cc:4841] TS c2057183ee0a4a74bda5cdbc575d3ca0 (127.5.61.1:37073): tablet d9d7f861ec9e4be0b6910d9ca02af7fc (table test-table [id=5cb54dea633e4b118bf6aef2e8a5c024]) successfully deleted
I20260809 20:19:31.742569 18911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:19:31.706689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44232 (local address 127.5.61.1:37073)
0809 20:19:31.706917 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 20:19:31.706926 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:19:31.717673 (+ 10747us) server_negotiation.cc:407] Connection header received
0809 20:19:31.717918 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:19:31.717924 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:19:31.718007 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:19:31.718119 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:19:31.720269 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.721538 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.727894 (+ 6356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.728650 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.728771 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:19:31.734899 (+ 6128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:19:31.734915 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:19:31.734934 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:19:31.734977 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:19:31.739513 (+ 4536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:19:31.740455 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:19:31.740461 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:19:31.740464 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:19:31.740608 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:19:31.741074 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:19:31.741077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:19:31.741079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:19:31.741463 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:19:31.741678 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 20:19:31.742078 (+ 400us) server_negotiation.cc:299] Negotiation successful
0809 20:19:31.742269 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 20:19:31.743257 18587 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:19:31.841084 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 18445
I20260809 20:19:31.861864 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 18598
Aug 09 20:19:31 dist-test-slave-6k22 krb5kdc[17826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.2@KRBTEST.COM
I20260809 20:19:31.877034 18915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:19:31.862665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50504 (local address 127.5.61.2:45149)
0809 20:19:31.862849 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 20:19:31.862855 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:19:31.862963 (+ 108us) server_negotiation.cc:407] Connection header received
0809 20:19:31.863190 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:19:31.863195 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:19:31.863260 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:19:31.863375 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:19:31.864681 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.865425 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.866752 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.867123 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.867260 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:19:31.871814 (+ 4554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:19:31.871825 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:19:31.871839 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:19:31.871886 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:19:31.874498 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:19:31.875096 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:19:31.875103 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:19:31.875105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:19:31.875251 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:19:31.875658 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:19:31.875661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:19:31.875663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:19:31.876024 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:19:31.876228 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 20:19:31.876513 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 20:19:31.876667 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 20:19:31.877678 18740 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:19:31.970556 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 18598
I20260809 20:19:31.990485 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 18751
Aug 09 20:19:31 dist-test-slave-6k22 krb5kdc[17826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.3@KRBTEST.COM
I20260809 20:19:32.006148 18914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:19:31.991403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58006 (local address 127.5.61.3:46531)
0809 20:19:31.991623 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 20:19:31.991628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:19:31.991650 (+ 22us) server_negotiation.cc:407] Connection header received
0809 20:19:31.991849 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:19:31.991853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:19:31.991922 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:19:31.992043 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:19:31.993483 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.994224 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.995794 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:19:31.996201 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:19:31.996335 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:19:32.000757 (+ 4422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:19:32.000770 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:19:32.000786 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:19:32.000817 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:19:32.003344 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:19:32.004252 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:19:32.004262 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:19:32.004265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:19:32.004446 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:19:32.004918 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:19:32.004922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:19:32.004924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:19:32.005290 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:19:32.005524 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 20:19:32.005722 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 20:19:32.005892 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 20:19:32.006788 18891 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:19:32.096367 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 18751
I20260809 20:19:32.115442 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 18319
I20260809 20:19:32.116696 18388 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:19:32.230459 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 18319
I20260809 20:19:32.255822 5364 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 20:19:32.602169 5364 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 20:19:32.602499 5364 mini_ranger.cc:66] Stopping Ranger...
I20260809 20:19:32.947739 5364 mini_ranger.cc:68] Stopped Ranger
2026-08-09 20:19:32.948 UTC [17863] LOG: received smart shutdown request
2026-08-09 20:19:32.952 UTC [17863] LOG: background worker "logical replication launcher" (PID 17873) exited with exit code 1
2026-08-09 20:19:32.952 UTC [17867] LOG: shutting down
2026-08-09 20:19:33.123 UTC [17863] LOG: database system is shut down
2026-08-09T20:19:33Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50297 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:19:33 dist-test-slave-6k22 krb5kdc[18963](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:19:33 dist-test-slave-6k22 krb5kdc[18963](info): set up 2 sockets
Aug 09 20:19:33 dist-test-slave-6k22 krb5kdc[18963](info): commencing operation
krb5kdc: starting...
W20260809 20:19:34.425289 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:19:34 dist-test-slave-6k22 krb5kdc[18963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:19:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:19:34Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.61.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.61.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.61.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/rangeradmin_127.5.61.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.61.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/rangeradmin_127.5.61.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.61.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.61.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.61.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/rangerlookup_127.5.61.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.61.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/rangerlookup_127.5.61.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.61.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.61.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.61.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/HTTP_127.5.61.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.61.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/HTTP_127.5.61.20@KRBTEST.COM.keytab.
I20260809 20:19:34.775719 5364 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task7uf_Cz/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/postgres -l logfile start
2026-08-09 20:19:37.138 UTC [18996] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 20:19:37.138 UTC [18996] LOG: listening on IPv4 address "127.5.61.20", port 44643
2026-08-09 20:19:37.141 UTC [18996] LOG: listening on Unix socket "/tmp/.s.PGSQL.44643"
2026-08-09 20:19:37.164 UTC [18999] LOG: database system was shut down at 2026-08-09 20:19:35 UTC
2026-08-09 20:19:37.168 UTC [18996] LOG: database system is ready to accept connections
I20260809 20:19:38.205163 5364 mini_postgres.cc:96] Postgres bound to 44643
2026-08-09 20:19:38.231 UTC [19007] FATAL: database "slave" does not exist
127.5.61.20:44643 - accepting connections
I20260809 20:19:38.231971 5364 mini_ranger.cc:161] Starting Ranger...
I20260809 20:19:38.259231 5364 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 20:19:38.708062 5364 mini_ranger.cc:87] Created ranger Postgres database
I20260809 20:19:38.708158 5364 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ranger-admin
2026-08-09 20:19:38,996 [I] DB FLAVOR :POSTGRES
2026-08-09 20:19:38,997 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 20:19:38,997 [I] Checking connection..
2026-08-09 20:19:38,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:44643/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 20:19:39,253 [I] Checking connection passed.
2026-08-09 20:19:39,253 [I] --------- Verifying version history table ---------
2026-08-09 20:19:39,253 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:44643/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 20:19:39,514 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 20:19:39,515 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:44643/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 20:19:39,751 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 20:19:39,752 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 20:19:39,752 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:44643/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 20:19:40.008 UTC [19081] WARNING: there is no transaction in progress
2026-08-09 20:19:40,016 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 20:19:40,016 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:44643/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 20:19:40,282 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 20:19:40,283 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 20:19:40,283 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:44643/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 20:19:40,543 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:44643/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 20:19:40,780 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:44643/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6k22.c.gcp-upstream.internal','N') ;"
2026-08-09 20:19:41,034 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:44643/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 20:19:41,278 [I] Table x_portal_user does not exist in database ranger
2026-08-09 20:19:41,279 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:44643/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 20:19:41,531 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 20:19:41,532 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:44643/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 20:19:41,788 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:44643/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 20:19:42,013 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 20:19:42,013 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:44643/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 20:19:42.952 UTC [19198] WARNING: there is no transaction in progress
2026-08-09 20:19:42.969 UTC [19198] WARNING: there is no transaction in progress
2026-08-09 20:19:42.985 UTC [19198] WARNING: there is no transaction in progress
2026-08-09 20:19:43.000 UTC [19198] WARNING: there is no transaction in progress
2026-08-09 20:19:43.015 UTC [19198] WARNING: there is no transaction in progress
2026-08-09 20:19:43.032 UTC [19198] WARNING: there is no transaction in progress
2026-08-09 20:19:43.085 UTC [19198] WARNING: there is no transaction in progress
2026-08-09 20:19:43.092 UTC [19198] WARNING: there is no transaction in progress
2026-08-09 20:19:43.107 UTC [19198] WARNING: there is no transaction in progress
2026-08-09 20:19:43.117 UTC [19198] WARNING: there is no transaction in progress
2026-08-09 20:19:43.126 UTC [19198] WARNING: there is no transaction in progress
2026-08-09 20:19:43.138 UTC [19198] WARNING: there is no transaction in progress
2026-08-09 20:19:43.148 UTC [19198] WARNING: there is no transaction in progress
2026-08-09 20:19:43.155 UTC [19198] WARNING: there is no transaction in progress
2026-08-09 20:19:43.161 UTC [19198] WARNING: there is no transaction in progress
2026-08-09 20:19:43.168 UTC [19198] WARNING: there is no transaction in progress
2026-08-09 20:19:43,777 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 20:19:43,778 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:44643/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 20:19:44,028 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 20:19:44,029 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:44643/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 20:19:44,299 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 20:19:44,299 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:44643/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 20:19:44,544 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 20:19:44,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:44643/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 20:19:44,788 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:44643/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 20:19:45,042 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:44643/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6k22.c.gcp-upstream.internal';"
2026-08-09 20:19:45,290 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 20:19:45,290 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:44643/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 20:19:45,543 [I] DB_PATCHES have already been applied
I20260809 20:19:45.548916 5364 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ranger-admin:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task7uf_Cz/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 20:19:45.549001 5364 mini_ranger.cc:193] Using host: 127.5.61.20
I20260809 20:19:45.567610 5364 mini_ranger.cc:227] Ranger admin URL: http://127.5.61.20:56173
Aug 09, 2026 8:19:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 8:19:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 8:19:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 8:19:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 8:19:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 8:19:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 8:19:46 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 20:19:46.798139 5364 mini_ranger.cc:160] Time spent starting Ranger: real 8.566s user 0.008s sys 0.041s
Aug 09, 2026 8:20:06 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 8:20:06 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 8:20:06 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 8:20:06 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 8:20:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 8:20:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 8:20:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 8:20:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 8:20:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 8:20:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 8:20:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 8:20:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 8:20:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 8:20:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 8:20:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 8:20:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 8:20:06 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 8:20:07 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 20:20:07.812503 5364 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.61.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.61.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.61.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/rangerkms_127.5.61.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.61.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/rangerkms_127.5.61.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.61.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.61.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.61.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/HTTP_127.5.61.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.61.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/HTTP_127.5.61.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 20:20:07 dist-test-slave-6k22 krb5kdc[18963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306807, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 20:20:07.997792 5364 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 20:20:08.021932 5364 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 20:20:08.892802 5364 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 20:20:08.892927 5364 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ranger-kms
I20260809 20:20:08.892984 5364 mini_ranger_kms.cc:227] Using postgres at 127.5.61.20:44643
2026-08-09 20:20:09,262 [I] DB FLAVOR :POSTGRES
2026-08-09 20:20:09,262 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 20:20:09,262 [I] Checking connection
2026-08-09 20:20:09,262 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:44643/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 20:20:09,490 [I] connection success
2026-08-09 20:20:09,491 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 20:20:09,491 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 20:20:09,491 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:44643/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 20:20:09,731 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 20:20:09,731 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 20:20:09,731 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 20:20:09,732 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.61.20:44643/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 20:20:09,997 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 20:20:10.469834 5364 mini_ranger_kms.cc:310] Created kms service
I20260809 20:20:10.622131 5364 mini_ranger_kms.cc:326] Created kudu user
I20260809 20:20:10.679693 5364 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 20:20:10 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306807, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.61.20@KRBTEST.COM
I20260809 20:20:11.099691 5364 mini_ranger_kms.cc:384] Added ranger policy
I20260809 20:20:11.100009 5364 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ranger-kms:/tmp/dist-test-task7uf_Cz/build/asan/bin/postgresql.jar:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task7uf_Cz/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 20:20:11.100055 5364 mini_ranger_kms.cc:240] Using host: 127.5.61.20
I20260809 20:20:11.121338 5364 mini_ranger_kms.cc:276] Ranger KMS PID: 19410
I20260809 20:20:11.121462 5364 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.61.20:48427
Aug 09, 2026 8:20:12 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 8:20:12 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 8:20:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 8:20:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 8:20:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 8:20:12 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260809 20:20:13.592691 5364 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.595s user 0.058s sys 0.338s
I20260809 20:20:13.592962 5364 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 20:20:13.593118 5364 mini_ranger_kms.cc:401] 127.5.61.20:48427/kms/v1/keys
Aug 09 20:20:13 dist-test-slave-6k22 krb5kdc[18963](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786306813, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.61.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 20:20:13 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306813, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.61.20@KRBTEST.COM for HTTP/127.5.61.20@KRBTEST.COM
Aug 09 20:20:14 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306813, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.61.20@KRBTEST.COM for HTTP/127.5.61.20@KRBTEST.COM
Aug 09 20:20:16 dist-test-slave-6k22 krb5kdc[18963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:20:16.667795 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:38015
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:38015
--ranger_config_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.61.20:48427/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:20:16.995190 19466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:16.995462 19466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:16.995646 19466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:20:17.002530 19466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:20:17.002652 19466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:17.002760 19466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:17.002866 19466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:20:17.002948 19466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:20:17.012519 19466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.61.20:48427/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:38015
--ranger_config_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:38015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:17.013811 19466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:17.015106 19466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:17.028491 19473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:17.028611 19475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:17.028463 19472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:17.029141 19466 server_base.cc:1034] running on GCE node
I20260809 20:20:17.029989 19466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:17.031358 19466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:17.032586 19466 hybrid_clock.cc:648] HybridClock initialized: now 1786306817032531 us; error 69 us; skew 500 ppm
I20260809 20:20:17.032943 19466 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:20:17 dist-test-slave-6k22 krb5kdc[18963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306817, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:17.040266 19466 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:20:17.041884 19466 webserver.cc:460] Webserver started at http://127.5.61.62:44603/ using document root <none> and password file <none>
I20260809 20:20:17.042537 19466 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:17.042675 19466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:20:17.042995 19466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 20:20:17 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306817, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for HTTP/127.5.61.20@KRBTEST.COM
I20260809 20:20:17.098555 19466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "069abe955bc4476abda6360d2bb8f917"
format_stamp: "Formatted at 2026-08-09 20:20:17 on dist-test-slave-6k22"
server_key: "ea25b7f0fe98b5b6545f398cb83308fe"
server_key_iv: "3a98cb10af54cb82fdfdbd24e435322e"
server_key_version: "kuduclusterkey@0"
I20260809 20:20:17.099440 19466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "069abe955bc4476abda6360d2bb8f917"
format_stamp: "Formatted at 2026-08-09 20:20:17 on dist-test-slave-6k22"
server_key: "ea25b7f0fe98b5b6545f398cb83308fe"
server_key_iv: "3a98cb10af54cb82fdfdbd24e435322e"
server_key_version: "kuduclusterkey@0"
I20260809 20:20:17.104993 19466 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 20:20:17.120909 19482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:17.122474 19466 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:20:17.122675 19466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "069abe955bc4476abda6360d2bb8f917"
format_stamp: "Formatted at 2026-08-09 20:20:17 on dist-test-slave-6k22"
server_key: "ea25b7f0fe98b5b6545f398cb83308fe"
server_key_iv: "3a98cb10af54cb82fdfdbd24e435322e"
server_key_version: "kuduclusterkey@0"
I20260809 20:20:17.122905 19466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:17.135414 19466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:17.139410 19466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:17.139784 19466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:17.159797 19466 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:38015
I20260809 20:20:17.159825 19538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:38015 every 8 connection(s)
I20260809 20:20:17.161343 19466 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:20:17.165424 19539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:20:17.165561 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 19466
I20260809 20:20:17.165815 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:20:17.166628 5364 external_mini_cluster.cc:1428] Setting key c00f9ddad4b29f9c7e7513a6921922d4
I20260809 20:20:17.185690 19539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917: Bootstrap starting.
I20260809 20:20:17.189973 19539 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917: Neither blocks nor log segments found. Creating new log.
I20260809 20:20:17.191371 19539 log.cc:824] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917: Log is configured to *not* fsync() on all Append() calls
I20260809 20:20:17.194618 19539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917: No bootstrap required, opened a new log
Aug 09 20:20:17 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:20:17.205140 19539 raft_consensus.cc:348] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069abe955bc4476abda6360d2bb8f917" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38015 } }
I20260809 20:20:17.205678 19539 raft_consensus.cc:374] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:20:17.205811 19539 raft_consensus.cc:729] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 069abe955bc4476abda6360d2bb8f917, State: Initialized, Role: FOLLOWER
I20260809 20:20:17.206530 19539 consensus_queue.cc:260] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069abe955bc4476abda6360d2bb8f917" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38015 } }
I20260809 20:20:17.206845 19539 raft_consensus.cc:388] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:20:17.207026 19539 raft_consensus.cc:482] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:20:17.207263 19539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:20:17.210647 19539 raft_consensus.cc:504] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069abe955bc4476abda6360d2bb8f917" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38015 } }
I20260809 20:20:17.211473 19539 leader_election.cc:302] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 069abe955bc4476abda6360d2bb8f917; no voters:
I20260809 20:20:17.212224 19539 leader_election.cc:288] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:20:17.212360 19544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:20:17.213546 19542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:17.168695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41146 (local address 127.5.61.62:38015)
0809 20:20:17.183705 (+ 15010us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:17.183721 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:17.183770 (+ 49us) server_negotiation.cc:407] Connection header received
0809 20:20:17.186428 (+ 2658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:17.186456 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:17.186934 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:17.187456 (+ 522us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:17.189798 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:17.191954 (+ 2156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:17.193670 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:17.194653 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:17.195488 (+ 835us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:17.202144 (+ 6656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:17.202154 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:17.202174 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:17.202220 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:17.207024 (+ 4804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:17.208089 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:17.208099 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:17.208113 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:17.208300 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:17.208776 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:17.208781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:17.208785 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:17.209516 (+ 731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:17.209842 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:17.211229 (+ 1387us) server_negotiation.cc:299] Negotiation successful
0809 20:20:17.211726 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14731,"thread_start_us":14537,"threads_started":1}
I20260809 20:20:17.218423 19544 raft_consensus.cc:686] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [term 1 LEADER]: Becoming Leader. State: Replica: 069abe955bc4476abda6360d2bb8f917, State: Running, Role: LEADER
I20260809 20:20:17.218969 19544 consensus_queue.cc:237] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069abe955bc4476abda6360d2bb8f917" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38015 } }
I20260809 20:20:17.221961 19539 sys_catalog.cc:564] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:20:17.221891 19546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "069abe955bc4476abda6360d2bb8f917" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069abe955bc4476abda6360d2bb8f917" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38015 } } }
I20260809 20:20:17.222260 19546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [sys.catalog]: This master's current role is: LEADER
I20260809 20:20:17.224843 19548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:20:17.226939 19545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 069abe955bc4476abda6360d2bb8f917. Latest consensus state: current_term: 1 leader_uuid: "069abe955bc4476abda6360d2bb8f917" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069abe955bc4476abda6360d2bb8f917" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38015 } } }
I20260809 20:20:17.227620 19545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [sys.catalog]: This master's current role is: LEADER
I20260809 20:20:17.231272 19548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:20:17.248116 19548 catalog_manager.cc:1347] Generated new cluster ID: fbcc2061e2ec4b3ebd16c580be14c1dc
I20260809 20:20:17.248289 19548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:20:17.269608 19548 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:20:17.271337 19548 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:20:17.291646 19548 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917: Generated new TSK 0
I20260809 20:20:17.292846 19548 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:20:17.295142 19539 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
WARNING: no policy specified for kudu/127.5.61.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
I20260809 20:20:18.185061 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.61.20:48427/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:38015
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 20:20:18 dist-test-slave-6k22 krb5kdc[18963](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786306818, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 20:20:18.619966 19588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:18.620337 19588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:18.620548 19588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:20:18.630007 19588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:18.630136 19588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:18.630318 19588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:20:18.640398 19588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.61.20:48427/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:38015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:18.641825 19588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:18.643167 19588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:18.668349 19598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:18.669342 19601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:18.673831 19588 server_base.cc:1034] running on GCE node
W20260809 20:20:18.674629 19599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:18.675539 19588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:18.676717 19588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:18.677991 19588 hybrid_clock.cc:648] HybridClock initialized: now 1786306818677922 us; error 58 us; skew 500 ppm
I20260809 20:20:18.678413 19588 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:20:18 dist-test-slave-6k22 krb5kdc[18963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306818, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:18.682302 19588 init.cc:377] Logged in from keytab as kudu/127.5.61.1@KRBTEST.COM (short username kudu)
I20260809 20:20:18.688153 19588 webserver.cc:460] Webserver started at http://127.5.61.1:33037/ using document root <none> and password file <none>
I20260809 20:20:18.689137 19588 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:18.689284 19588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:20:18.689709 19588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 20:20:18 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306818, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for HTTP/127.5.61.20@KRBTEST.COM
I20260809 20:20:18.709180 19588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed"
format_stamp: "Formatted at 2026-08-09 20:20:18 on dist-test-slave-6k22"
server_key: "8c79e9d77d27733202297aa32dfe2f17"
server_key_iv: "d187be18350b3a43c28d9b838f96dfd0"
server_key_version: "kuduclusterkey@0"
I20260809 20:20:18.710497 19588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed"
format_stamp: "Formatted at 2026-08-09 20:20:18 on dist-test-slave-6k22"
server_key: "8c79e9d77d27733202297aa32dfe2f17"
server_key_iv: "d187be18350b3a43c28d9b838f96dfd0"
server_key_version: "kuduclusterkey@0"
I20260809 20:20:18.723966 19588 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.004s sys 0.008s
I20260809 20:20:18.754382 19609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:18.756470 19588 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.006s sys 0.003s
I20260809 20:20:18.756747 19588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed"
format_stamp: "Formatted at 2026-08-09 20:20:18 on dist-test-slave-6k22"
server_key: "8c79e9d77d27733202297aa32dfe2f17"
server_key_iv: "d187be18350b3a43c28d9b838f96dfd0"
server_key_version: "kuduclusterkey@0"
I20260809 20:20:18.757120 19588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:18.794677 19588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:18.799945 19588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:18.800275 19588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:18.801467 19588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:20:18.803192 19588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:20:18.803289 19588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:18.803462 19588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:20:18.803529 19588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 20:20:18 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306818, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for HTTP/127.5.61.20@KRBTEST.COM
Aug 09 20:20:19 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306818, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for HTTP/127.5.61.20@KRBTEST.COM
I20260809 20:20:19.420955 19728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:34449 every 8 connection(s)
I20260809 20:20:19.421355 19588 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:34449
I20260809 20:20:19.423225 19588 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:20:19.429471 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 19588
I20260809 20:20:19.429737 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:20:19.430687 5364 external_mini_cluster.cc:1428] Setting key a653c3fd570d59182803508907d4053d
Aug 09 20:20:19 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306818, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:20:19.480885 19731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:19.433075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:51303 (local address 127.5.61.62:38015)
0809 20:20:19.458279 (+ 25204us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:19.458287 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:19.458314 (+ 27us) server_negotiation.cc:407] Connection header received
0809 20:20:19.458407 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:19.458414 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:19.458505 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:19.458647 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:19.460971 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:19.462260 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:19.466373 (+ 4113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:19.467095 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:19.467220 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:19.472348 (+ 5128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:19.472356 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:19.472362 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:19.472422 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:19.476574 (+ 4152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:19.477647 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:19.477654 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:19.477658 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:19.477848 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:19.478492 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:19.478498 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:19.478502 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:19.478905 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:19.479063 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:19.480145 (+ 1082us) server_negotiation.cc:299] Negotiation successful
0809 20:20:19.480400 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25035,"thread_start_us":30474,"threads_started":1}
I20260809 20:20:19.483495 19732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:19.432916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38015 (local address 127.5.61.1:51303)
0809 20:20:19.447146 (+ 14230us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:19.447208 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:19.449898 (+ 2690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:19.459146 (+ 9248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:19.459160 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:19.459872 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:20:19.460736 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:19.460762 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:19.463377 (+ 2615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:19.463383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:19.465442 (+ 2059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:19.465458 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:19.467679 (+ 2221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:19.467690 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:19.468009 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:19.468923 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:20:19.468959 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:20:19.472066 (+ 3107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:20:19.476807 (+ 4741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:19.476817 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:19.476836 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:19.477468 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:19.478027 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:19.478035 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:19.478039 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:19.478302 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:19.479145 (+ 843us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:20:19.479154 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:20:19.479636 (+ 482us) client_negotiation.cc:769] Sending connection context
0809 20:20:19.481415 (+ 1779us) client_negotiation.cc:240] Negotiation successful
0809 20:20:19.482015 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13858,"thread_start_us":17275,"threads_started":1}
I20260809 20:20:19.486166 19729 heartbeater.cc:347] Connected to a master server at 127.5.61.62:38015
I20260809 20:20:19.486517 19729 heartbeater.cc:464] Registering TS with master...
I20260809 20:20:19.487412 19729 heartbeater.cc:511] Master 127.5.61.62:38015 requested a full tablet report, sending...
I20260809 20:20:19.490289 19501 ts_manager.cc:194] Registered new tserver with Master: 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed (127.5.61.1:34449)
I20260809 20:20:19.492776 19501 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.1@KRBTEST.COM'} at 127.5.61.1:51303
WARNING: no policy specified for kudu/127.5.61.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
W20260809 20:20:19.565229 5364 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.5.61.2: real 0.131s user 0.000s sys 0.026s
I20260809 20:20:19.566403 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.61.20:48427/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:38015
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:20:20.035418 19735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:20.035773 19735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:20.035940 19735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:20:20.043005 19735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:20.043164 19735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:20.043334 19735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:20:20.052851 19735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.61.20:48427/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:38015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:20.054010 19735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:20.055284 19735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:20.068097 19747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:20.067984 19748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:20.068094 19750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:20.069344 19735 server_base.cc:1034] running on GCE node
I20260809 20:20:20.069931 19735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:20.070784 19735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:20.071974 19735 hybrid_clock.cc:648] HybridClock initialized: now 1786306820071927 us; error 43 us; skew 500 ppm
I20260809 20:20:20.072227 19735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:20:20 dist-test-slave-6k22 krb5kdc[18963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306820, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:20.075171 19735 init.cc:377] Logged in from keytab as kudu/127.5.61.2@KRBTEST.COM (short username kudu)
I20260809 20:20:20.076545 19735 webserver.cc:460] Webserver started at http://127.5.61.2:39303/ using document root <none> and password file <none>
I20260809 20:20:20.077191 19735 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:20.077334 19735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:20:20.077629 19735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 20:20:20 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306820, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for HTTP/127.5.61.20@KRBTEST.COM
I20260809 20:20:20.090940 19735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "1bff06e732924c3da56a84607266dbc3"
format_stamp: "Formatted at 2026-08-09 20:20:20 on dist-test-slave-6k22"
server_key: "ed3eb849e3e46bf5d2b345920cea373f"
server_key_iv: "ab38bf228323085b824f3ac78ff7a744"
server_key_version: "kuduclusterkey@0"
I20260809 20:20:20.091830 19735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "1bff06e732924c3da56a84607266dbc3"
format_stamp: "Formatted at 2026-08-09 20:20:20 on dist-test-slave-6k22"
server_key: "ed3eb849e3e46bf5d2b345920cea373f"
server_key_iv: "ab38bf228323085b824f3ac78ff7a744"
server_key_version: "kuduclusterkey@0"
I20260809 20:20:20.097237 19735 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260809 20:20:20.111330 19757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:20.112836 19735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 20:20:20.113005 19735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "1bff06e732924c3da56a84607266dbc3"
format_stamp: "Formatted at 2026-08-09 20:20:20 on dist-test-slave-6k22"
server_key: "ed3eb849e3e46bf5d2b345920cea373f"
server_key_iv: "ab38bf228323085b824f3ac78ff7a744"
server_key_version: "kuduclusterkey@0"
I20260809 20:20:20.113196 19735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:20.128067 19735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:20.131801 19735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:20.132087 19735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:20.133069 19735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:20:20.134438 19735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:20:20.134510 19735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:20.134639 19735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:20:20.134696 19735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:20.193727 19735 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:37847
I20260809 20:20:20.193756 19881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:37847 every 8 connection(s)
I20260809 20:20:20.195545 19735 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:20:20.198173 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 19735
I20260809 20:20:20.198364 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:20:20.199056 5364 external_mini_cluster.cc:1428] Setting key c7149263c9ce41dff8996fb826c01d15
Aug 09 20:20:20 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306820, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:20:20.224634 19886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:20.200411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38015 (local address 127.5.61.2:38603)
0809 20:20:20.201675 (+ 1264us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:20.201718 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:20.203912 (+ 2194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:20.204892 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:20.204907 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:20.205544 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:20:20.206356 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:20.206382 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:20.208012 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:20.208020 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:20.210207 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:20.210225 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:20.211481 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:20.211490 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:20.211887 (+ 397us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:20.212657 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:20:20.212690 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:20:20.215674 (+ 2984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:20:20.219746 (+ 4072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:20.219757 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:20.219774 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:20.220374 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:20.220802 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:20.220810 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:20.220816 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:20.221064 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:20.221840 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:20:20.221851 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:20:20.222272 (+ 421us) client_negotiation.cc:769] Sending connection context
0809 20:20:20.222598 (+ 326us) client_negotiation.cc:240] Negotiation successful
0809 20:20:20.223123 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"thread_start_us":512,"threads_started":1}
I20260809 20:20:20.225947 19884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:20.198927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:38603 (local address 127.5.61.62:38015)
0809 20:20:20.202266 (+ 3339us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:20.202272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:20.202293 (+ 21us) server_negotiation.cc:407] Connection header received
0809 20:20:20.204299 (+ 2006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:20.204307 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:20.204405 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:20.204500 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:20.206553 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:20.207829 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:20.210384 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:20.211064 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:20.211199 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:20.215907 (+ 4708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:20.215913 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:20.215918 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:20.215962 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:20.219605 (+ 3643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:20.220507 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:20.220515 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:20.220521 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:20.220654 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:20.221194 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:20.221202 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:20.221207 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:20.221579 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:20.221718 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:20.225198 (+ 3480us) server_negotiation.cc:299] Negotiation successful
0809 20:20:20.225440 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3175,"thread_start_us":3073,"threads_started":1}
I20260809 20:20:20.227902 19882 heartbeater.cc:347] Connected to a master server at 127.5.61.62:38015
I20260809 20:20:20.228230 19882 heartbeater.cc:464] Registering TS with master...
I20260809 20:20:20.228952 19882 heartbeater.cc:511] Master 127.5.61.62:38015 requested a full tablet report, sending...
I20260809 20:20:20.230540 19501 ts_manager.cc:194] Registered new tserver with Master: 1bff06e732924c3da56a84607266dbc3 (127.5.61.2:37847)
I20260809 20:20:20.231720 19501 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.2@KRBTEST.COM'} at 127.5.61.2:38603
WARNING: no policy specified for kudu/127.5.61.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
I20260809 20:20:20.268949 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.61.20:48427/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:38015
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 20:20:20.496758 19729 heartbeater.cc:503] Master 127.5.61.62:38015 was elected leader, sending a full tablet report...
W20260809 20:20:20.600589 19889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:20.600893 19889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:20.601068 19889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:20:20.608126 19889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:20.608278 19889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:20.608446 19889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:20:20.618230 19889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.61.20:48427/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:38015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:20.619535 19889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:20.620899 19889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:20.634029 19895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:20.634006 19894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:20.634536 19897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:20.634959 19889 server_base.cc:1034] running on GCE node
I20260809 20:20:20.635644 19889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:20.636576 19889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:20.637774 19889 hybrid_clock.cc:648] HybridClock initialized: now 1786306820637740 us; error 23 us; skew 500 ppm
I20260809 20:20:20.638103 19889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:20:20 dist-test-slave-6k22 krb5kdc[18963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306820, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:20.641386 19889 init.cc:377] Logged in from keytab as kudu/127.5.61.3@KRBTEST.COM (short username kudu)
I20260809 20:20:20.642868 19889 webserver.cc:460] Webserver started at http://127.5.61.3:36429/ using document root <none> and password file <none>
I20260809 20:20:20.643549 19889 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:20.643746 19889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:20:20.644057 19889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 20:20:20 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306820, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for HTTP/127.5.61.20@KRBTEST.COM
I20260809 20:20:20.660457 19889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "3b6eff229fe64a648411389f47c0db01"
format_stamp: "Formatted at 2026-08-09 20:20:20 on dist-test-slave-6k22"
server_key: "319a1aa6b0d421b464cdc68491aeeeb1"
server_key_iv: "aea4d416b7231bd1bdeb705b8cf94755"
server_key_version: "kuduclusterkey@0"
I20260809 20:20:20.661293 19889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "3b6eff229fe64a648411389f47c0db01"
format_stamp: "Formatted at 2026-08-09 20:20:20 on dist-test-slave-6k22"
server_key: "319a1aa6b0d421b464cdc68491aeeeb1"
server_key_iv: "aea4d416b7231bd1bdeb705b8cf94755"
server_key_version: "kuduclusterkey@0"
I20260809 20:20:20.666941 19889 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 20:20:20.683956 19904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:20.685704 19889 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260809 20:20:20.685895 19889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "3b6eff229fe64a648411389f47c0db01"
format_stamp: "Formatted at 2026-08-09 20:20:20 on dist-test-slave-6k22"
server_key: "319a1aa6b0d421b464cdc68491aeeeb1"
server_key_iv: "aea4d416b7231bd1bdeb705b8cf94755"
server_key_version: "kuduclusterkey@0"
I20260809 20:20:20.686144 19889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:20.721673 19889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:20.727151 19889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:20.727603 19889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:20.728868 19889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:20:20.730693 19889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:20:20.730799 19889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:20.730948 19889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:20:20.731019 19889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:20.767525 19889 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:33305
I20260809 20:20:20.767611 20029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:33305 every 8 connection(s)
I20260809 20:20:20.768999 19889 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:20:20.774005 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 19889
I20260809 20:20:20.774169 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:20:20.774659 5364 external_mini_cluster.cc:1428] Setting key 1bb0308c9afe0b9e4ee7ecaebb84c49b
Aug 09 20:20:20 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306820, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:20:20.788318 20033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:20.771596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:37039 (local address 127.5.61.62:38015)
0809 20:20:20.772034 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:20.772038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:20.773017 (+ 979us) server_negotiation.cc:407] Connection header received
0809 20:20:20.774907 (+ 1890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:20.774913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:20.775003 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:20.775134 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:20.776721 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:20.777486 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:20.779052 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:20.779488 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:20.779658 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:20.782781 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:20.782804 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:20.782807 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:20.782843 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:20.785351 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:20.786059 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:20.786067 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:20.786072 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:20.786199 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:20.786630 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:20.786636 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:20.786640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:20.786914 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:20.787077 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:20.787729 (+ 652us) server_negotiation.cc:299] Negotiation successful
0809 20:20:20.787952 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":222,"threads_started":1}
I20260809 20:20:20.789294 20034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:20.772111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38015 (local address 127.5.61.3:37039)
0809 20:20:20.772855 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:20.772891 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:20.774587 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:20.775357 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:20.775371 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:20.775963 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:20:20.776550 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:20.776571 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:20.777644 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:20.777654 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:20.778899 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:20.778911 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:20.779745 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:20.779764 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:20.779995 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:20.780558 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:20:20.780578 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:20:20.782570 (+ 1992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:20:20.785507 (+ 2937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:20.785515 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:20.785528 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:20.785936 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:20.786309 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:20.786314 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:20.786318 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:20.786514 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:20.787052 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:20:20.787060 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:20:20.787355 (+ 295us) client_negotiation.cc:769] Sending connection context
0809 20:20:20.787640 (+ 285us) client_negotiation.cc:240] Negotiation successful
0809 20:20:20.788080 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":311,"threads_started":1}
I20260809 20:20:20.790535 20030 heartbeater.cc:347] Connected to a master server at 127.5.61.62:38015
I20260809 20:20:20.790817 20030 heartbeater.cc:464] Registering TS with master...
I20260809 20:20:20.791635 20030 heartbeater.cc:511] Master 127.5.61.62:38015 requested a full tablet report, sending...
I20260809 20:20:20.792860 19501 ts_manager.cc:194] Registered new tserver with Master: 3b6eff229fe64a648411389f47c0db01 (127.5.61.3:33305)
I20260809 20:20:20.793602 19501 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.3@KRBTEST.COM'} at 127.5.61.3:37039
I20260809 20:20:20.801802 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 20:20:20.803369 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 19588
Aug 09 20:20:20 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.1@KRBTEST.COM
I20260809 20:20:20.818874 20036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:20.804312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43488 (local address 127.5.61.1:34449)
0809 20:20:20.804937 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:20.804942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:20.804964 (+ 22us) server_negotiation.cc:407] Connection header received
0809 20:20:20.805018 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:20.805024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:20.805185 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:20.805301 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:20.806639 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:20.807695 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:20.809029 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:20.809522 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:20.809617 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:20.813648 (+ 4031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:20.813659 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:20.813670 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:20.813705 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:20.816381 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:20.816992 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:20.816997 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:20.817000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:20.817188 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:20.817532 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:20.817535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:20.817536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:20.817837 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:20.818064 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:20.818282 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 20:20:20.818501 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":328,"threads_started":1}
I20260809 20:20:20.820065 19723 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:20:20.903007 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 19588
I20260809 20:20:20.922829 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 19735
Aug 09 20:20:20 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.2@KRBTEST.COM
I20260809 20:20:20.938359 20038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:20.923772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35080 (local address 127.5.61.2:37847)
0809 20:20:20.924331 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:20.924337 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:20.924360 (+ 23us) server_negotiation.cc:407] Connection header received
0809 20:20:20.924417 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:20.924423 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:20.924693 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:20.924866 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:20.926227 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:20.927246 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:20.928745 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:20.929257 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:20.929399 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:20.933446 (+ 4047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:20.933456 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:20.933468 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:20.933506 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:20.936058 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:20.936668 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:20.936675 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:20.936677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:20.936839 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:20.937155 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:20.937158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:20.937160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:20.937464 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:20.937672 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:20.937898 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 20:20:20.938075 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":282,"threads_started":1}
I20260809 20:20:20.939719 19875 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:20:21.021181 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 19735
I20260809 20:20:21.040691 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 19889
Aug 09 20:20:21 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.3@KRBTEST.COM
I20260809 20:20:21.056779 20041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:21.041607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34688 (local address 127.5.61.3:33305)
0809 20:20:21.042214 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:21.042221 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:21.042249 (+ 28us) server_negotiation.cc:407] Connection header received
0809 20:20:21.042308 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:21.042314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:21.042544 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:21.042868 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:21.044306 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:21.045318 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:21.046725 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:21.047300 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:21.047434 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:21.051599 (+ 4165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:21.051616 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:21.051629 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:21.051659 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:21.054171 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:21.054875 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:21.054881 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:21.054883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:21.054992 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:21.055310 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:21.055313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:21.055314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:21.055662 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:21.055921 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:21.056206 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 20:20:21.056388 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":293,"threads_started":1}
I20260809 20:20:21.058036 20022 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:20:21.139694 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 19889
I20260809 20:20:21.158309 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 19466
I20260809 20:20:21.159679 19533 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:20:21.265501 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 19466
I20260809 20:20:21.291476 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:38015
--webserver_interface=127.5.61.62
--webserver_port=44603
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:38015
--ranger_config_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.61.20:48427/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.61.20:48427/kms with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:20:21.617944 20044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:21.618216 20044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:21.618432 20044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:20:21.625492 20044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:20:21.625626 20044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:21.625721 20044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:21.625810 20044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:20:21.625881 20044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:20:21.635475 20044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.61.20:48427/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:38015
--ranger_config_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:38015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=44603
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:21.636682 20044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:21.637996 20044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:21.649709 20050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:21.649760 20051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:21.649989 20053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:21.650919 20044 server_base.cc:1034] running on GCE node
I20260809 20:20:21.651742 20044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:21.653057 20044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:21.654309 20044 hybrid_clock.cc:648] HybridClock initialized: now 1786306821654264 us; error 48 us; skew 500 ppm
I20260809 20:20:21.654661 20044 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:20:21 dist-test-slave-6k22 krb5kdc[18963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306821, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:21.657716 20044 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:20:21.659118 20044 webserver.cc:460] Webserver started at http://127.5.61.62:44603/ using document root <none> and password file <none>
I20260809 20:20:21.659845 20044 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:21.659997 20044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 20:20:21 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306821, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for HTTP/127.5.61.20@KRBTEST.COM
I20260809 20:20:21.714783 20044 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 20:20:21.717913 20061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:21.719306 20044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 20:20:21.719503 20044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "069abe955bc4476abda6360d2bb8f917"
format_stamp: "Formatted at 2026-08-09 20:20:17 on dist-test-slave-6k22"
server_key: "ea25b7f0fe98b5b6545f398cb83308fe"
server_key_iv: "3a98cb10af54cb82fdfdbd24e435322e"
server_key_version: "kuduclusterkey@0"
I20260809 20:20:21.720528 20044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:21.743333 20044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:21.746850 20044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:21.747135 20044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:21.765575 20044 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:38015
I20260809 20:20:21.765597 20119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:38015 every 8 connection(s)
I20260809 20:20:21.767023 20044 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:20:21.773630 20120 sys_catalog.cc:263] Verifying existing consensus state
I20260809 20:20:21.776980 20120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917: Bootstrap starting.
I20260809 20:20:21.777287 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 20044
I20260809 20:20:21.778486 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.61.20:48427/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:34449
--local_ip_for_outbound_sockets=127.5.61.1
--tserver_master_addrs=127.5.61.62:38015
--webserver_port=33037
--webserver_interface=127.5.61.1
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.61.20:48427/kms with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 20:20:21.799295 20120 log.cc:824] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917: Log is configured to *not* fsync() on all Append() calls
I20260809 20:20:21.808161 20120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 20:20:21.808668 20120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917: Bootstrap complete.
I20260809 20:20:21.817384 20120 raft_consensus.cc:348] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069abe955bc4476abda6360d2bb8f917" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38015 } }
I20260809 20:20:21.818172 20120 raft_consensus.cc:729] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 069abe955bc4476abda6360d2bb8f917, State: Initialized, Role: FOLLOWER
I20260809 20:20:21.818812 20120 consensus_queue.cc:260] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069abe955bc4476abda6360d2bb8f917" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38015 } }
I20260809 20:20:21.819041 20120 raft_consensus.cc:388] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:20:21.819181 20120 raft_consensus.cc:482] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:20:21.819375 20120 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [term 1 FOLLOWER]: Advancing to term 2
I20260809 20:20:21.823199 20120 raft_consensus.cc:504] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069abe955bc4476abda6360d2bb8f917" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38015 } }
I20260809 20:20:21.823760 20120 leader_election.cc:302] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 069abe955bc4476abda6360d2bb8f917; no voters:
I20260809 20:20:21.824452 20120 leader_election.cc:288] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 20:20:21.824605 20124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 20:20:21.825927 20124 raft_consensus.cc:686] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [term 2 LEADER]: Becoming Leader. State: Replica: 069abe955bc4476abda6360d2bb8f917, State: Running, Role: LEADER
I20260809 20:20:21.826725 20120 sys_catalog.cc:564] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:20:21.826560 20124 consensus_queue.cc:237] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069abe955bc4476abda6360d2bb8f917" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38015 } }
I20260809 20:20:21.829169 20125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "069abe955bc4476abda6360d2bb8f917" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069abe955bc4476abda6360d2bb8f917" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38015 } } }
I20260809 20:20:21.829612 20125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [sys.catalog]: This master's current role is: LEADER
I20260809 20:20:21.831811 20126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 069abe955bc4476abda6360d2bb8f917. Latest consensus state: current_term: 2 leader_uuid: "069abe955bc4476abda6360d2bb8f917" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069abe955bc4476abda6360d2bb8f917" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38015 } } }
I20260809 20:20:21.832295 20126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917 [sys.catalog]: This master's current role is: LEADER
I20260809 20:20:21.833534 20130 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:20:21.840008 20130 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:20:21.841604 20130 catalog_manager.cc:1259] Loaded cluster ID: fbcc2061e2ec4b3ebd16c580be14c1dc
I20260809 20:20:21.841797 20130 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:20:21.845745 20130 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:20:21.847155 20130 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917: Loaded TSK: 0
I20260809 20:20:21.848676 20130 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:20:21.853523 20120 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260809 20:20:22.192649 20122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:22.193079 20122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:22.193286 20122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:20:22.203372 20122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:22.203560 20122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:22.203831 20122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:20:22.221983 20122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.61.20:48427/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:34449
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=33037
--tserver_master_addrs=127.5.61.62:38015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:22.223712 20122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:22.225365 20122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:22.261469 20164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:22.261849 20163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:22.266852 20122 server_base.cc:1034] running on GCE node
W20260809 20:20:22.267247 20166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:22.268149 20122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:22.269683 20122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:22.270972 20122 hybrid_clock.cc:648] HybridClock initialized: now 1786306822270911 us; error 56 us; skew 500 ppm
I20260809 20:20:22.271466 20122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:20:22 dist-test-slave-6k22 krb5kdc[18963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306822, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:22.279597 20122 init.cc:377] Logged in from keytab as kudu/127.5.61.1@KRBTEST.COM (short username kudu)
I20260809 20:20:22.292044 20122 webserver.cc:460] Webserver started at http://127.5.61.1:33037/ using document root <none> and password file <none>
I20260809 20:20:22.293020 20122 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:22.293248 20122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 20:20:22 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306822, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for HTTP/127.5.61.20@KRBTEST.COM
I20260809 20:20:22.334306 20122 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.000s sys 0.011s
I20260809 20:20:22.348223 20175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:22.355427 20122 fs_manager.cc:725] Time spent opening block manager: real 0.019s user 0.003s sys 0.001s
I20260809 20:20:22.355670 20122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed"
format_stamp: "Formatted at 2026-08-09 20:20:18 on dist-test-slave-6k22"
server_key: "8c79e9d77d27733202297aa32dfe2f17"
server_key_iv: "d187be18350b3a43c28d9b838f96dfd0"
server_key_version: "kuduclusterkey@0"
I20260809 20:20:22.357319 20122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:22.399334 20122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:22.404843 20122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:22.405264 20122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:22.412979 20122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:20:22.421402 20122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:20:22.421608 20122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:22.421837 20122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:20:22.421965 20122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:22.518234 20122 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:34449
I20260809 20:20:22.518285 20300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:34449 every 8 connection(s)
I20260809 20:20:22.520462 20122 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:20:22.525337 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 20122
I20260809 20:20:22.526414 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.61.20:48427/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:37847
--local_ip_for_outbound_sockets=127.5.61.2
--tserver_master_addrs=127.5.61.62:38015
--webserver_port=39303
--webserver_interface=127.5.61.2
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.61.20:48427/kms with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 20:20:22 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306822, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:20:22.566220 20303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:22.524171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38015 (local address 127.5.61.1:57189)
0809 20:20:22.535754 (+ 11583us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:22.535815 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:22.538615 (+ 2800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:22.543019 (+ 4404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:22.543029 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:22.543654 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:20:22.544500 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:22.544528 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:22.547012 (+ 2484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:22.547021 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:22.549463 (+ 2442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:22.549491 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:22.551474 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:22.551483 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:22.551835 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:22.552673 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:20:22.552709 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:20:22.555387 (+ 2678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:20:22.560390 (+ 5003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:22.560401 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:22.560415 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:22.561051 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:22.561652 (+ 601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:22.561657 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:22.561662 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:22.561935 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:22.563221 (+ 1286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:20:22.563229 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:20:22.563691 (+ 462us) client_negotiation.cc:769] Sending connection context
0809 20:20:22.564040 (+ 349us) client_negotiation.cc:240] Negotiation successful
0809 20:20:22.564681 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10343,"thread_start_us":10560,"threads_started":1}
I20260809 20:20:22.566917 20304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:22.524214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:57189 (local address 127.5.61.62:38015)
0809 20:20:22.539477 (+ 15263us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:22.539498 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:22.539550 (+ 52us) server_negotiation.cc:407] Connection header received
0809 20:20:22.540035 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:22.540057 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:22.540520 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:22.540908 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:22.544768 (+ 3860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:22.546823 (+ 2055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:22.549675 (+ 2852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:22.550846 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:22.551064 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:22.555762 (+ 4698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:22.555773 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:22.555792 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:22.555848 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:22.560196 (+ 4348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:22.561243 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:22.561253 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:22.561262 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:22.561473 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:22.562090 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:22.562096 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:22.562100 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:22.562882 (+ 782us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:22.563127 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:22.565017 (+ 1890us) server_negotiation.cc:299] Negotiation successful
0809 20:20:22.565488 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14986,"thread_start_us":14726,"threads_started":1}
I20260809 20:20:22.570716 20301 heartbeater.cc:347] Connected to a master server at 127.5.61.62:38015
I20260809 20:20:22.571070 20301 heartbeater.cc:464] Registering TS with master...
I20260809 20:20:22.572000 20301 heartbeater.cc:511] Master 127.5.61.62:38015 requested a full tablet report, sending...
W20260809 20:20:22.575480 20301 heartbeater.cc:644] Failed to heartbeat to 127.5.61.62:38015 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 20:20:22.575762 20301 heartbeater.cc:464] Registering TS with master...
I20260809 20:20:22.576061 20301 heartbeater.cc:511] Master 127.5.61.62:38015 requested a full tablet report, sending...
I20260809 20:20:22.577672 20301 heartbeater.cc:464] Registering TS with master...
I20260809 20:20:22.577966 20301 heartbeater.cc:511] Master 127.5.61.62:38015 requested a full tablet report, sending...
W20260809 20:20:22.579221 20301 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 20:20:22.930001 20305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:22.930310 20305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:22.930483 20305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:20:22.937845 20305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:22.937980 20305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:22.938158 20305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:20:22.947777 20305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.61.20:48427/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:37847
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=39303
--tserver_master_addrs=127.5.61.62:38015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:22.949009 20305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:22.950278 20305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:22.968263 20319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:22.969884 20318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:22.971392 20321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:22.972244 20305 server_base.cc:1034] running on GCE node
I20260809 20:20:22.972801 20305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:22.973888 20305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:22.975178 20305 hybrid_clock.cc:648] HybridClock initialized: now 1786306822975111 us; error 53 us; skew 500 ppm
I20260809 20:20:22.975591 20305 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:20:22 dist-test-slave-6k22 krb5kdc[18963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306822, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:22.979652 20305 init.cc:377] Logged in from keytab as kudu/127.5.61.2@KRBTEST.COM (short username kudu)
I20260809 20:20:22.981374 20305 webserver.cc:460] Webserver started at http://127.5.61.2:39303/ using document root <none> and password file <none>
I20260809 20:20:22.982069 20305 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:22.982225 20305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 20:20:22 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306822, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for HTTP/127.5.61.20@KRBTEST.COM
I20260809 20:20:23.002686 20305 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 20:20:23.006790 20331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:23.008432 20305 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260809 20:20:23.008610 20305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "1bff06e732924c3da56a84607266dbc3"
format_stamp: "Formatted at 2026-08-09 20:20:20 on dist-test-slave-6k22"
server_key: "ed3eb849e3e46bf5d2b345920cea373f"
server_key_iv: "ab38bf228323085b824f3ac78ff7a744"
server_key_version: "kuduclusterkey@0"
I20260809 20:20:23.010003 20305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:23.032099 20305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:23.037288 20305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:23.037647 20305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:23.038784 20305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:20:23.040254 20305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:20:23.040375 20305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:23.040515 20305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:20:23.040602 20305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:23.139917 20305 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:37847
I20260809 20:20:23.139999 20459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:37847 every 8 connection(s)
I20260809 20:20:23.141847 20305 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:20:23.151953 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 20305
I20260809 20:20:23.152706 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.61.20:48427/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:33305
--local_ip_for_outbound_sockets=127.5.61.3
--tserver_master_addrs=127.5.61.62:38015
--webserver_port=36429
--webserver_interface=127.5.61.3
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.61.20:48427/kms with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 20:20:23 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306822, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:20:23.181346 20465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:23.147325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:35871 (local address 127.5.61.62:38015)
0809 20:20:23.149339 (+ 2014us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:23.149346 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:23.156239 (+ 6893us) server_negotiation.cc:407] Connection header received
0809 20:20:23.156327 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:23.156335 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:23.156434 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:23.156547 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:23.158818 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:23.160053 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:23.162701 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:23.163533 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:23.163732 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:23.173141 (+ 9409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:23.173149 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:23.173157 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:23.173203 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:23.177022 (+ 3819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:23.178013 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:23.178021 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:23.178028 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:23.178198 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:23.178857 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:23.178865 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:23.178872 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:23.179300 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:23.179442 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:23.180590 (+ 1148us) server_negotiation.cc:299] Negotiation successful
0809 20:20:23.180867 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1882,"thread_start_us":1754,"threads_started":1}
I20260809 20:20:23.183715 20464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:23.146687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38015 (local address 127.5.61.2:35871)
0809 20:20:23.152123 (+ 5436us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:23.152169 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:23.154768 (+ 2599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:23.157025 (+ 2257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:23.157040 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:23.157740 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:20:23.158591 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:23.158620 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:23.160249 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:23.160257 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:23.162522 (+ 2265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:23.162548 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:23.164109 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:23.164118 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:23.164446 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:23.165263 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:20:23.165294 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:20:23.172875 (+ 7581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:20:23.177204 (+ 4329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:23.177214 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:23.177231 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:23.177861 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:23.178360 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:23.178367 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:23.178387 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:23.178693 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:23.179598 (+ 905us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:20:23.179608 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:20:23.180085 (+ 477us) client_negotiation.cc:769] Sending connection context
0809 20:20:23.181722 (+ 1637us) client_negotiation.cc:240] Negotiation successful
0809 20:20:23.182275 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5134,"thread_start_us":4918,"threads_started":1}
I20260809 20:20:23.185722 20461 heartbeater.cc:347] Connected to a master server at 127.5.61.62:38015
I20260809 20:20:23.186048 20461 heartbeater.cc:464] Registering TS with master...
I20260809 20:20:23.186887 20461 heartbeater.cc:511] Master 127.5.61.62:38015 requested a full tablet report, sending...
I20260809 20:20:23.189049 20082 ts_manager.cc:194] Registered new tserver with Master: 1bff06e732924c3da56a84607266dbc3 (127.5.61.2:37847)
Aug 09 20:20:23 dist-test-slave-6k22 krb5kdc[18963](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786306823, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:23.191223 20082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.2@KRBTEST.COM'} at 127.5.61.2:35871
I20260809 20:20:23.579380 20301 heartbeater.cc:464] Registering TS with master...
I20260809 20:20:23.579900 20301 heartbeater.cc:511] Master 127.5.61.62:38015 requested a full tablet report, sending...
I20260809 20:20:23.581372 20082 ts_manager.cc:194] Registered new tserver with Master: 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed (127.5.61.1:34449)
I20260809 20:20:23.582458 20082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.1@KRBTEST.COM'} at 127.5.61.1:57189
W20260809 20:20:23.640307 20466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:23.640756 20466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:23.641006 20466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
Aug 09 20:20:23 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306823, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for HTTP/127.5.61.20@KRBTEST.COM
W20260809 20:20:23.653484 20466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:23.653690 20466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:23.653920 20466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:20:23.670202 20466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.61.20:48427/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:33305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=36429
--tserver_master_addrs=127.5.61.62:38015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:23.672502 20466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:23.674232 20466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:23.694193 20472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:23.708235 20473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:23.713711 20475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:23.725525 20466 server_base.cc:1034] running on GCE node
I20260809 20:20:23.726652 20466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:23.734051 20466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:23.737592 20466 hybrid_clock.cc:648] HybridClock initialized: now 1786306823737495 us; error 99 us; skew 500 ppm
I20260809 20:20:23.738147 20466 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:20:23 dist-test-slave-6k22 krb5kdc[18963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306823, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:23.742858 20466 init.cc:377] Logged in from keytab as kudu/127.5.61.3@KRBTEST.COM (short username kudu)
I20260809 20:20:23.768596 20466 webserver.cc:460] Webserver started at http://127.5.61.3:36429/ using document root <none> and password file <none>
I20260809 20:20:23.769845 20466 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:23.770117 20466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 20:20:23 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306823, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for HTTP/127.5.61.20@KRBTEST.COM
I20260809 20:20:23.813987 20466 fs_manager.cc:709] Time spent opening directory manager: real 0.018s user 0.008s sys 0.008s
I20260809 20:20:23.824417 20485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:23.826467 20466 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.001s sys 0.007s
I20260809 20:20:23.826741 20466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "3b6eff229fe64a648411389f47c0db01"
format_stamp: "Formatted at 2026-08-09 20:20:20 on dist-test-slave-6k22"
server_key: "319a1aa6b0d421b464cdc68491aeeeb1"
server_key_iv: "aea4d416b7231bd1bdeb705b8cf94755"
server_key_version: "kuduclusterkey@0"
I20260809 20:20:23.828327 20466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:23.889639 20466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:23.895211 20466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:23.895690 20466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:23.903172 20466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:20:23.911331 20466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:20:23.911531 20466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 20:20:23.911753 20466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:20:23.911887 20466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 20:20:24 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306823, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for HTTP/127.5.61.20@KRBTEST.COM
I20260809 20:20:24.195735 20461 heartbeater.cc:503] Master 127.5.61.62:38015 was elected leader, sending a full tablet report...
I20260809 20:20:24.405370 20466 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:33305
I20260809 20:20:24.405442 20617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:33305 every 8 connection(s)
I20260809 20:20:24.407274 20466 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:20:24.408730 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 20466
Aug 09 20:20:24 dist-test-slave-6k22 krb5kdc[18963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306823, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:20:24.450181 20620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:24.409765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55226 (local address 127.5.61.62:38015)
0809 20:20:24.427847 (+ 18082us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:24.427856 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:24.430797 (+ 2941us) server_negotiation.cc:407] Connection header received
0809 20:20:24.430877 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:24.430882 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:24.430957 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:24.431065 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:24.433769 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.434843 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.437657 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.438331 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.438491 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:24.442522 (+ 4031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:24.442528 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:24.442533 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:24.442569 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:24.446271 (+ 3702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:24.447124 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:24.447130 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:24.447134 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:24.447321 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:24.447811 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:24.447816 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:24.447820 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:24.448172 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:24.448432 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:24.449561 (+ 1129us) server_negotiation.cc:299] Negotiation successful
0809 20:20:24.449775 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17918,"thread_start_us":18094,"threads_started":1}
I20260809 20:20:24.453557 20622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:24.412192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:38015 (local address 127.5.61.3:55109)
0809 20:20:24.423153 (+ 10961us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:24.423203 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:24.432244 (+ 9041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:24.433290 (+ 1046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:24.433302 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:24.435218 (+ 1916us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:20:24.436081 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:24.436113 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.437703 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.437708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:24.439681 (+ 1973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:24.439695 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.440917 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.440924 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:24.441188 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:24.441880 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:20:24.441908 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:20:24.444902 (+ 2994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:20:24.448919 (+ 4017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:24.448927 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:24.448943 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:24.449493 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:24.449906 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:24.449911 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:24.449915 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:24.450170 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:24.451087 (+ 917us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:20:24.451095 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:20:24.451470 (+ 375us) client_negotiation.cc:769] Sending connection context
0809 20:20:24.451800 (+ 330us) client_negotiation.cc:240] Negotiation successful
0809 20:20:24.452294 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10654,"thread_start_us":10423,"threads_started":1}
I20260809 20:20:24.454587 20623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:24.412946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:55109 (local address 127.5.61.62:38015)
0809 20:20:24.429349 (+ 16403us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:24.429355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:24.429378 (+ 23us) server_negotiation.cc:407] Connection header received
0809 20:20:24.432670 (+ 3292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:24.432676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:24.432739 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:24.432832 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:24.436332 (+ 3500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.437515 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.439881 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.440494 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.440602 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:24.445220 (+ 4618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:24.445230 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:24.445235 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:24.445287 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:24.448767 (+ 3480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:24.449635 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:24.449641 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:24.449644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:24.449784 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:24.450552 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:24.450558 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:24.450561 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:24.450888 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:24.451003 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:24.454045 (+ 3042us) server_negotiation.cc:299] Negotiation successful
0809 20:20:24.454237 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16281,"thread_start_us":16412,"threads_started":1}
I20260809 20:20:24.456470 20618 heartbeater.cc:347] Connected to a master server at 127.5.61.62:38015
I20260809 20:20:24.456777 20618 heartbeater.cc:464] Registering TS with master...
I20260809 20:20:24.457559 20618 heartbeater.cc:511] Master 127.5.61.62:38015 requested a full tablet report, sending...
I20260809 20:20:24.459092 20082 ts_manager.cc:194] Registered new tserver with Master: 3b6eff229fe64a648411389f47c0db01 (127.5.61.3:33305)
I20260809 20:20:24.460245 20082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.3@KRBTEST.COM'} at 127.5.61.3:55109
I20260809 20:20:24.465054 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 20:20:24.488137 20623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:24.468564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55242 (local address 127.5.61.62:38015)
0809 20:20:24.468729 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:24.468734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:24.472117 (+ 3383us) server_negotiation.cc:407] Connection header received
0809 20:20:24.472284 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:24.472289 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:24.472393 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:24.472500 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:24.474424 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.475507 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.477419 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.478099 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.478231 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:24.481546 (+ 3315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:24.481553 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:24.481558 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:24.481596 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:24.485185 (+ 3589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:24.485995 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:24.486002 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:24.486005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:24.486180 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:24.486602 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:24.486607 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:24.486611 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:24.486965 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:24.487222 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:24.487555 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 20:20:24.487803 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 20:20:24.494391 20082 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55242:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:20:24.505883 20082 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:20:24.559250 20635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:24.522430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43496 (local address 127.5.61.1:34449)
0809 20:20:24.549259 (+ 26829us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:24.549269 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:24.549297 (+ 28us) server_negotiation.cc:407] Connection header received
0809 20:20:24.549404 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:24.549412 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:24.549710 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:24.549913 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:20:24.552056 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.553569 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.556148 (+ 2579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.557623 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.557794 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:24.558039 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:24.558789 (+ 750us) server_negotiation.cc:299] Negotiation successful
0809 20:20:24.558946 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26642,"thread_start_us":26485,"threads_started":1}
I20260809 20:20:24.559312 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:24.522297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:33305 (local address 127.0.0.1:34694)
0809 20:20:24.549492 (+ 27195us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:24.549510 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:24.549619 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:24.550717 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:24.550724 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:24.550756 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:20:24.551107 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:24.551120 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.553404 (+ 2284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.553409 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:24.555962 (+ 2553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:24.555981 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.557869 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.557875 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:24.558523 (+ 648us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:24.558552 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 20:20:24.558641 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 20:20:24.558833 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27083,"mutex_wait_us":403,"thread_start_us":23155,"threads_started":1}
I20260809 20:20:24.560272 20634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:24.522130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:34449 (local address 127.0.0.1:43496)
0809 20:20:24.545873 (+ 23743us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:24.545935 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:24.547923 (+ 1988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:24.550167 (+ 2244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:24.550174 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:24.550202 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:20:24.550688 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:24.550703 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.553745 (+ 3042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.553750 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:24.555964 (+ 2214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:24.555981 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.558205 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.558212 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:24.558425 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:24.558511 (+ 86us) client_negotiation.cc:769] Sending connection context
0809 20:20:24.558642 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 20:20:24.559878 (+ 1236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23459,"mutex_wait_us":1053,"thread_start_us":28040,"threads_started":1}
I20260809 20:20:24.561627 20636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:24.522924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34694 (local address 127.5.61.3:33305)
0809 20:20:24.549858 (+ 26934us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:24.549866 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:24.549898 (+ 32us) server_negotiation.cc:407] Connection header received
0809 20:20:24.549990 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:24.549998 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:24.550222 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:24.550408 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:20:24.551324 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.553253 (+ 1929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.556184 (+ 2931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.557623 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.558059 (+ 436us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:24.558328 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:24.560795 (+ 2467us) server_negotiation.cc:299] Negotiation successful
0809 20:20:24.561290 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26745,"thread_start_us":30021,"threads_started":1}
I20260809 20:20:24.562517 20637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:24.546715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35090 (local address 127.5.61.2:37847)
0809 20:20:24.548063 (+ 1348us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:24.548072 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:24.550138 (+ 2066us) server_negotiation.cc:407] Connection header received
0809 20:20:24.550622 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:24.550630 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:24.550827 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:24.550969 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:20:24.555672 (+ 4703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.557229 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.559764 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.561174 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.561280 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:24.561474 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:24.562038 (+ 564us) server_negotiation.cc:299] Negotiation successful
0809 20:20:24.562216 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1179,"thread_start_us":1027,"threads_started":1}
I20260809 20:20:24.563328 20633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:24.548102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:37847 (local address 127.0.0.1:35090)
0809 20:20:24.550041 (+ 1939us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:24.550057 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:24.550441 (+ 384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:24.551241 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:24.551245 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:24.551269 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:20:24.551739 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:24.551751 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.557406 (+ 5655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.557411 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:24.559582 (+ 2171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:24.559600 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.561663 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.561670 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:24.561841 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:24.561873 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 20:20:24.562867 (+ 994us) client_negotiation.cc:240] Negotiation successful
0809 20:20:24.562988 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1839,"mutex_wait_us":2,"thread_start_us":1146,"threads_started":1}
I20260809 20:20:24.564690 20231 tablet_service.cc:1467] Processing CreateTablet for tablet f836519e0cb34a31a0788938b147faaf (DEFAULT_TABLE table=test-table [id=d6a545f47f2f479e942b1826c90c3b1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:20:24.566318 20231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f836519e0cb34a31a0788938b147faaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:20:24.566808 20388 tablet_service.cc:1467] Processing CreateTablet for tablet f836519e0cb34a31a0788938b147faaf (DEFAULT_TABLE table=test-table [id=d6a545f47f2f479e942b1826c90c3b1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:20:24.566900 20539 tablet_service.cc:1467] Processing CreateTablet for tablet f836519e0cb34a31a0788938b147faaf (DEFAULT_TABLE table=test-table [id=d6a545f47f2f479e942b1826c90c3b1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:20:24.568181 20388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f836519e0cb34a31a0788938b147faaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:20:24.568310 20539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f836519e0cb34a31a0788938b147faaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:20:24.585731 20301 heartbeater.cc:503] Master 127.5.61.62:38015 was elected leader, sending a full tablet report...
I20260809 20:20:24.596392 20642 tablet_bootstrap.cc:492] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed: Bootstrap starting.
I20260809 20:20:24.602721 20640 tablet_bootstrap.cc:492] T f836519e0cb34a31a0788938b147faaf P 1bff06e732924c3da56a84607266dbc3: Bootstrap starting.
I20260809 20:20:24.603637 20641 tablet_bootstrap.cc:492] T f836519e0cb34a31a0788938b147faaf P 3b6eff229fe64a648411389f47c0db01: Bootstrap starting.
I20260809 20:20:24.607169 20641 tablet_bootstrap.cc:654] T f836519e0cb34a31a0788938b147faaf P 3b6eff229fe64a648411389f47c0db01: Neither blocks nor log segments found. Creating new log.
I20260809 20:20:24.608461 20641 log.cc:824] T f836519e0cb34a31a0788938b147faaf P 3b6eff229fe64a648411389f47c0db01: Log is configured to *not* fsync() on all Append() calls
I20260809 20:20:24.610728 20640 tablet_bootstrap.cc:654] T f836519e0cb34a31a0788938b147faaf P 1bff06e732924c3da56a84607266dbc3: Neither blocks nor log segments found. Creating new log.
I20260809 20:20:24.611807 20642 tablet_bootstrap.cc:654] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed: Neither blocks nor log segments found. Creating new log.
I20260809 20:20:24.613054 20642 log.cc:824] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed: Log is configured to *not* fsync() on all Append() calls
I20260809 20:20:24.615142 20640 log.cc:824] T f836519e0cb34a31a0788938b147faaf P 1bff06e732924c3da56a84607266dbc3: Log is configured to *not* fsync() on all Append() calls
I20260809 20:20:24.621994 20640 tablet_bootstrap.cc:492] T f836519e0cb34a31a0788938b147faaf P 1bff06e732924c3da56a84607266dbc3: No bootstrap required, opened a new log
I20260809 20:20:24.624603 20642 tablet_bootstrap.cc:492] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed: No bootstrap required, opened a new log
I20260809 20:20:24.624900 20642 ts_tablet_manager.cc:1397] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed: Time spent bootstrapping tablet: real 0.029s user 0.008s sys 0.002s
I20260809 20:20:24.627236 20640 ts_tablet_manager.cc:1397] T f836519e0cb34a31a0788938b147faaf P 1bff06e732924c3da56a84607266dbc3: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.004s
I20260809 20:20:24.637907 20641 tablet_bootstrap.cc:492] T f836519e0cb34a31a0788938b147faaf P 3b6eff229fe64a648411389f47c0db01: No bootstrap required, opened a new log
I20260809 20:20:24.638258 20641 ts_tablet_manager.cc:1397] T f836519e0cb34a31a0788938b147faaf P 3b6eff229fe64a648411389f47c0db01: Time spent bootstrapping tablet: real 0.035s user 0.012s sys 0.021s
I20260809 20:20:24.643383 20642 raft_consensus.cc:348] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 34449 } } peers { permanent_uuid: "3b6eff229fe64a648411389f47c0db01" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33305 } } peers { permanent_uuid: "1bff06e732924c3da56a84607266dbc3" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37847 } }
I20260809 20:20:24.644006 20642 raft_consensus.cc:374] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:20:24.644174 20642 raft_consensus.cc:729] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed, State: Initialized, Role: FOLLOWER
I20260809 20:20:24.644980 20642 consensus_queue.cc:260] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 34449 } } peers { permanent_uuid: "3b6eff229fe64a648411389f47c0db01" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33305 } } peers { permanent_uuid: "1bff06e732924c3da56a84607266dbc3" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37847 } }
I20260809 20:20:24.651191 20640 raft_consensus.cc:348] T f836519e0cb34a31a0788938b147faaf P 1bff06e732924c3da56a84607266dbc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 34449 } } peers { permanent_uuid: "3b6eff229fe64a648411389f47c0db01" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33305 } } peers { permanent_uuid: "1bff06e732924c3da56a84607266dbc3" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37847 } }
I20260809 20:20:24.651726 20640 raft_consensus.cc:374] T f836519e0cb34a31a0788938b147faaf P 1bff06e732924c3da56a84607266dbc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:20:24.651865 20640 raft_consensus.cc:729] T f836519e0cb34a31a0788938b147faaf P 1bff06e732924c3da56a84607266dbc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bff06e732924c3da56a84607266dbc3, State: Initialized, Role: FOLLOWER
I20260809 20:20:24.652604 20640 consensus_queue.cc:260] T f836519e0cb34a31a0788938b147faaf P 1bff06e732924c3da56a84607266dbc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 34449 } } peers { permanent_uuid: "3b6eff229fe64a648411389f47c0db01" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33305 } } peers { permanent_uuid: "1bff06e732924c3da56a84607266dbc3" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37847 } }
I20260809 20:20:24.653609 20642 ts_tablet_manager.cc:1428] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed: Time spent starting tablet: real 0.029s user 0.004s sys 0.015s
I20260809 20:20:24.653983 20641 raft_consensus.cc:348] T f836519e0cb34a31a0788938b147faaf P 3b6eff229fe64a648411389f47c0db01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 34449 } } peers { permanent_uuid: "3b6eff229fe64a648411389f47c0db01" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33305 } } peers { permanent_uuid: "1bff06e732924c3da56a84607266dbc3" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37847 } }
I20260809 20:20:24.654417 20641 raft_consensus.cc:374] T f836519e0cb34a31a0788938b147faaf P 3b6eff229fe64a648411389f47c0db01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:20:24.654526 20641 raft_consensus.cc:729] T f836519e0cb34a31a0788938b147faaf P 3b6eff229fe64a648411389f47c0db01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b6eff229fe64a648411389f47c0db01, State: Initialized, Role: FOLLOWER
I20260809 20:20:24.655229 20641 consensus_queue.cc:260] T f836519e0cb34a31a0788938b147faaf P 3b6eff229fe64a648411389f47c0db01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 34449 } } peers { permanent_uuid: "3b6eff229fe64a648411389f47c0db01" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33305 } } peers { permanent_uuid: "1bff06e732924c3da56a84607266dbc3" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37847 } }
I20260809 20:20:24.661621 20640 ts_tablet_manager.cc:1428] T f836519e0cb34a31a0788938b147faaf P 1bff06e732924c3da56a84607266dbc3: Time spent starting tablet: real 0.034s user 0.001s sys 0.014s
I20260809 20:20:24.665371 20618 heartbeater.cc:503] Master 127.5.61.62:38015 was elected leader, sending a full tablet report...
I20260809 20:20:24.667204 20641 ts_tablet_manager.cc:1428] T f836519e0cb34a31a0788938b147faaf P 3b6eff229fe64a648411389f47c0db01: Time spent starting tablet: real 0.029s user 0.017s sys 0.000s
W20260809 20:20:24.681018 20619 tablet.cc:2369] T f836519e0cb34a31a0788938b147faaf P 3b6eff229fe64a648411389f47c0db01: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 20:20:24.795127 20302 tablet.cc:2369] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:20:24.872339 20646 raft_consensus.cc:482] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:20:24.872659 20646 raft_consensus.cc:504] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 34449 } } peers { permanent_uuid: "3b6eff229fe64a648411389f47c0db01" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33305 } } peers { permanent_uuid: "1bff06e732924c3da56a84607266dbc3" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37847 } }
I20260809 20:20:24.874394 20646 leader_election.cc:288] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b6eff229fe64a648411389f47c0db01 (127.5.61.3:33305), 1bff06e732924c3da56a84607266dbc3 (127.5.61.2:37847)
I20260809 20:20:24.881840 20636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:24.874773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:39947 (local address 127.5.61.3:33305)
0809 20:20:24.875056 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:24.875066 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:24.875720 (+ 654us) server_negotiation.cc:407] Connection header received
0809 20:20:24.876076 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:24.876087 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:24.876189 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:24.876269 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:20:24.877170 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.878118 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.880141 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.880927 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.881060 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:24.881171 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:24.881454 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 20:20:24.881587 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260809 20:20:24.881906 20649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:24.874773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:33305 (local address 127.5.61.1:39947)
0809 20:20:24.875618 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:24.875639 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:24.875844 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:24.876510 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:24.876513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:24.876539 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:20:24.876998 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:24.877011 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.878290 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.878296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:24.879984 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:24.879997 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.881125 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.881130 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:24.881281 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:24.881314 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 20:20:24.881406 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 20:20:24.881526 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"mutex_wait_us":1,"spinlock_wait_cycles":24832,"thread_start_us":489,"threads_started":1}
I20260809 20:20:24.883003 20560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f836519e0cb34a31a0788938b147faaf" candidate_uuid: "4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b6eff229fe64a648411389f47c0db01" is_pre_election: true
I20260809 20:20:24.883239 20637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:24.875922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:44457 (local address 127.5.61.2:37847)
0809 20:20:24.876111 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:24.876116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:24.876689 (+ 573us) server_negotiation.cc:407] Connection header received
0809 20:20:24.876890 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:24.876897 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:24.876992 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:24.877096 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:20:24.877645 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.878674 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.880977 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.882114 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.882250 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:24.882335 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:24.882893 (+ 558us) server_negotiation.cc:299] Negotiation successful
0809 20:20:24.883012 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 20:20:24.883311 20650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:24.875765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:37847 (local address 127.5.61.1:44457)
0809 20:20:24.876551 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:24.876568 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:24.876733 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:24.877121 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:24.877125 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:24.877147 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:20:24.877480 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:24.877489 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.878829 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.878834 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:24.880810 (+ 1976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:24.880824 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.882539 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.882548 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:24.882712 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:24.882750 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 20:20:24.882854 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 20:20:24.882967 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":511,"threads_started":1}
I20260809 20:20:24.883635 20560 raft_consensus.cc:2454] T f836519e0cb34a31a0788938b147faaf P 3b6eff229fe64a648411389f47c0db01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed in term 0.
I20260809 20:20:24.884347 20411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f836519e0cb34a31a0788938b147faaf" candidate_uuid: "4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bff06e732924c3da56a84607266dbc3" is_pre_election: true
I20260809 20:20:24.884480 20179 leader_election.cc:302] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b6eff229fe64a648411389f47c0db01, 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed; no voters:
I20260809 20:20:24.884833 20411 raft_consensus.cc:2454] T f836519e0cb34a31a0788938b147faaf P 1bff06e732924c3da56a84607266dbc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed in term 0.
I20260809 20:20:24.884914 20646 raft_consensus.cc:2781] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:20:24.885058 20646 raft_consensus.cc:482] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:20:24.885155 20646 raft_consensus.cc:3037] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:20:24.887733 20646 raft_consensus.cc:504] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 34449 } } peers { permanent_uuid: "3b6eff229fe64a648411389f47c0db01" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33305 } } peers { permanent_uuid: "1bff06e732924c3da56a84607266dbc3" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37847 } }
I20260809 20:20:24.888365 20646 leader_election.cc:288] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed [CANDIDATE]: Term 1 election: Requested vote from peers 3b6eff229fe64a648411389f47c0db01 (127.5.61.3:33305), 1bff06e732924c3da56a84607266dbc3 (127.5.61.2:37847)
I20260809 20:20:24.888805 20560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f836519e0cb34a31a0788938b147faaf" candidate_uuid: "4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b6eff229fe64a648411389f47c0db01"
I20260809 20:20:24.888952 20411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f836519e0cb34a31a0788938b147faaf" candidate_uuid: "4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bff06e732924c3da56a84607266dbc3"
I20260809 20:20:24.889014 20560 raft_consensus.cc:3037] T f836519e0cb34a31a0788938b147faaf P 3b6eff229fe64a648411389f47c0db01 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:20:24.889144 20411 raft_consensus.cc:3037] T f836519e0cb34a31a0788938b147faaf P 1bff06e732924c3da56a84607266dbc3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:20:24.891322 20560 raft_consensus.cc:2454] T f836519e0cb34a31a0788938b147faaf P 3b6eff229fe64a648411389f47c0db01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed in term 1.
I20260809 20:20:24.891736 20179 leader_election.cc:302] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b6eff229fe64a648411389f47c0db01, 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed; no voters:
I20260809 20:20:24.892004 20646 raft_consensus.cc:2781] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:20:24.892370 20411 raft_consensus.cc:2454] T f836519e0cb34a31a0788938b147faaf P 1bff06e732924c3da56a84607266dbc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed in term 1.
I20260809 20:20:24.892759 20646 raft_consensus.cc:686] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed [term 1 LEADER]: Becoming Leader. State: Replica: 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed, State: Running, Role: LEADER
I20260809 20:20:24.893314 20646 consensus_queue.cc:237] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 34449 } } peers { permanent_uuid: "3b6eff229fe64a648411389f47c0db01" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33305 } } peers { permanent_uuid: "1bff06e732924c3da56a84607266dbc3" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37847 } }
I20260809 20:20:24.898141 20082 catalog_manager.cc:5495] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed (127.5.61.1). New cstate: current_term: 1 leader_uuid: "4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 34449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b6eff229fe64a648411389f47c0db01" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bff06e732924c3da56a84607266dbc3" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37847 } health_report { overall_health: UNKNOWN } } }
W20260809 20:20:24.902226 20462 tablet.cc:2369] T f836519e0cb34a31a0788938b147faaf P 1bff06e732924c3da56a84607266dbc3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:20:24.986536 20635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:24.981083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43500 (local address 127.5.61.1:34449)
0809 20:20:24.981280 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:24.981286 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:24.981305 (+ 19us) server_negotiation.cc:407] Connection header received
0809 20:20:24.981507 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:24.981511 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:24.981597 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:24.981751 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:20:24.982138 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.982856 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.984424 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:24.984812 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:24.984948 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:24.985436 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:20:24.985951 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:20:24.986124 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:24.986181 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 20:20:24.986261 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 20:20:24.994491 20411 raft_consensus.cc:1261] T f836519e0cb34a31a0788938b147faaf P 1bff06e732924c3da56a84607266dbc3 [term 1 FOLLOWER]: Refusing update from remote peer 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:20:24.994551 20560 raft_consensus.cc:1261] T f836519e0cb34a31a0788938b147faaf P 3b6eff229fe64a648411389f47c0db01 [term 1 FOLLOWER]: Refusing update from remote peer 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:20:24.995389 20651 consensus_queue.cc:1035] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bff06e732924c3da56a84607266dbc3" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:20:24.995913 20646 consensus_queue.cc:1035] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b6eff229fe64a648411389f47c0db01" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 33305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:20:25.010399 20657 mvcc.cc:205] Tried to move back new op lower bound from 7316712755167846400 to 7316712754766991360. Current Snapshot: MvccSnapshot[applied={T|T < 7316712755167846400}]
I20260809 20:20:25.018440 20081 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:20:25.018766 20081 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:20:25.022419 20081 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 069abe955bc4476abda6360d2bb8f917: Sending DeleteTablet for 3 replicas of tablet f836519e0cb34a31a0788938b147faaf
I20260809 20:20:25.023540 20231 tablet_service.cc:1514] Processing DeleteTablet for tablet f836519e0cb34a31a0788938b147faaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:20:25 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:43496
I20260809 20:20:25.023798 20388 tablet_service.cc:1514] Processing DeleteTablet for tablet f836519e0cb34a31a0788938b147faaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:20:25 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:35090
I20260809 20:20:25.024024 20539 tablet_service.cc:1514] Processing DeleteTablet for tablet f836519e0cb34a31a0788938b147faaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:20:25 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:34694
I20260809 20:20:25.026924 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 20122
I20260809 20:20:25.038152 20664 tablet_replica.cc:330] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed: stopping tablet replica
I20260809 20:20:25.039242 20666 tablet_replica.cc:330] T f836519e0cb34a31a0788938b147faaf P 3b6eff229fe64a648411389f47c0db01: stopping tablet replica
I20260809 20:20:25.039855 20666 raft_consensus.cc:2229] T f836519e0cb34a31a0788938b147faaf P 3b6eff229fe64a648411389f47c0db01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:20:25.040448 20666 raft_consensus.cc:2258] T f836519e0cb34a31a0788938b147faaf P 3b6eff229fe64a648411389f47c0db01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:20:25.039541 20664 raft_consensus.cc:2229] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:20:25.041363 20665 tablet_replica.cc:330] T f836519e0cb34a31a0788938b147faaf P 1bff06e732924c3da56a84607266dbc3: stopping tablet replica
I20260809 20:20:25.041700 20664 raft_consensus.cc:2258] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:20:25.041844 20665 raft_consensus.cc:2229] T f836519e0cb34a31a0788938b147faaf P 1bff06e732924c3da56a84607266dbc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:20:25.042369 20665 raft_consensus.cc:2258] T f836519e0cb34a31a0788938b147faaf P 1bff06e732924c3da56a84607266dbc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:20:25.044706 20666 ts_tablet_manager.cc:1905] T f836519e0cb34a31a0788938b147faaf P 3b6eff229fe64a648411389f47c0db01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:20:25.044755 20664 ts_tablet_manager.cc:1905] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:20:25.049100 20665 ts_tablet_manager.cc:1905] T f836519e0cb34a31a0788938b147faaf P 1bff06e732924c3da56a84607266dbc3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:20:25.051846 20666 ts_tablet_manager.cc:1918] T f836519e0cb34a31a0788938b147faaf P 3b6eff229fe64a648411389f47c0db01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:20:25.052083 20666 log.cc:1196] T f836519e0cb34a31a0788938b147faaf P 3b6eff229fe64a648411389f47c0db01: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/f836519e0cb34a31a0788938b147faaf
I20260809 20:20:25.052769 20666 ts_tablet_manager.cc:1939] T f836519e0cb34a31a0788938b147faaf P 3b6eff229fe64a648411389f47c0db01: Deleting consensus metadata
I20260809 20:20:25.054880 20064 catalog_manager.cc:4841] TS 3b6eff229fe64a648411389f47c0db01 (127.5.61.3:33305): tablet f836519e0cb34a31a0788938b147faaf (table test-table [id=d6a545f47f2f479e942b1826c90c3b1b]) successfully deleted
I20260809 20:20:25.057654 20664 ts_tablet_manager.cc:1918] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:20:25.057873 20664 log.cc:1196] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/f836519e0cb34a31a0788938b147faaf
I20260809 20:20:25.058490 20664 ts_tablet_manager.cc:1939] T f836519e0cb34a31a0788938b147faaf P 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed: Deleting consensus metadata
I20260809 20:20:25.060495 20665 ts_tablet_manager.cc:1918] T f836519e0cb34a31a0788938b147faaf P 1bff06e732924c3da56a84607266dbc3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:20:25.060802 20665 log.cc:1196] T f836519e0cb34a31a0788938b147faaf P 1bff06e732924c3da56a84607266dbc3: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/f836519e0cb34a31a0788938b147faaf
I20260809 20:20:25.061442 20665 ts_tablet_manager.cc:1939] T f836519e0cb34a31a0788938b147faaf P 1bff06e732924c3da56a84607266dbc3: Deleting consensus metadata
I20260809 20:20:25.061686 20065 catalog_manager.cc:4841] TS 4e31b9d9cdcb4e6bb8f1fe6fc2acc5ed (127.5.61.1:34449): tablet f836519e0cb34a31a0788938b147faaf (table test-table [id=d6a545f47f2f479e942b1826c90c3b1b]) successfully deleted
I20260809 20:20:25.063238 20635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:25.027836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43514 (local address 127.5.61.1:34449)
0809 20:20:25.028143 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:25.028152 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:25.037821 (+ 9669us) server_negotiation.cc:407] Connection header received
0809 20:20:25.038666 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:25.038675 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:25.038796 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:25.038887 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:25.042722 (+ 3835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:25.044036 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:25.049925 (+ 5889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:25.050610 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:25.050753 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:25.055309 (+ 4556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:25.055323 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:25.055347 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:25.055393 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:25.059784 (+ 4391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:25.060658 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:25.060666 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:25.060669 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:25.060855 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:25.061318 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:25.061323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:25.061326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:25.061775 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:25.062075 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:25.062465 (+ 390us) server_negotiation.cc:299] Negotiation successful
0809 20:20:25.062703 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260809 20:20:25.063251 20064 catalog_manager.cc:4841] TS 1bff06e732924c3da56a84607266dbc3 (127.5.61.2:37847): tablet f836519e0cb34a31a0788938b147faaf (table test-table [id=d6a545f47f2f479e942b1826c90c3b1b]) successfully deleted
I20260809 20:20:25.064107 20296 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:20:25.169552 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 20122
I20260809 20:20:25.187184 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 20305
I20260809 20:20:25.200182 20637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:25.188054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35092 (local address 127.5.61.2:37847)
0809 20:20:25.188226 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:25.188230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:25.188250 (+ 20us) server_negotiation.cc:407] Connection header received
0809 20:20:25.188378 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:25.188381 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:25.188443 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:25.188523 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:25.189808 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:25.190538 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:25.191887 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:25.192372 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:25.192487 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:25.194807 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:25.194819 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:25.194833 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:25.194862 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:25.197711 (+ 2849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:25.198293 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:25.198299 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:25.198301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:25.198462 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:25.198791 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:25.198794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:25.198795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:25.199132 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:25.199360 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:25.199659 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 20:20:25.199812 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 20:20:25.200789 20455 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:20:25.291030 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 20305
I20260809 20:20:25.311800 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 20466
I20260809 20:20:25.325517 20636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:25.312637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34702 (local address 127.5.61.3:33305)
0809 20:20:25.312838 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:25.312843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:25.312890 (+ 47us) server_negotiation.cc:407] Connection header received
0809 20:20:25.313070 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:25.313074 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:25.313137 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:25.313219 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:25.314608 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:25.315342 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:25.316661 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:25.317048 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:25.317143 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:25.320035 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:25.320048 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:25.320064 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:25.320093 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:25.322599 (+ 2506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:25.323294 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:25.323302 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:25.323304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:25.323475 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:25.323897 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:25.323903 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:25.323906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:25.324261 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:25.324617 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:25.324942 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 20:20:25.325168 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 20:20:25.326170 20612 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:20:25.412725 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 20466
I20260809 20:20:25.432590 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 20044
I20260809 20:20:25.433785 20114 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:20:25.546856 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 20044
I20260809 20:20:25.573613 5364 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 20:20:25.918411 5364 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 20:20:25.918605 5364 mini_ranger.cc:66] Stopping Ranger...
I20260809 20:20:26.263418 5364 mini_ranger.cc:68] Stopped Ranger
2026-08-09 20:20:26.263 UTC [18996] LOG: received smart shutdown request
2026-08-09 20:20:26.267 UTC [18996] LOG: background worker "logical replication launcher" (PID 19005) exited with exit code 1
2026-08-09 20:20:26.267 UTC [19000] LOG: shutting down
2026-08-09 20:20:26.432 UTC [18996] LOG: database system is shut down
2026-08-09T20:20:26Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53310 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:20:26 dist-test-slave-6k22 krb5kdc[20684](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:20:26 dist-test-slave-6k22 krb5kdc[20684](info): set up 2 sockets
Aug 09 20:20:26 dist-test-slave-6k22 krb5kdc[20684](info): commencing operation
krb5kdc: starting...
W20260809 20:20:27.734870 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:20:27 dist-test-slave-6k22 krb5kdc[20684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:20:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:20:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:20:27.981074 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:39377
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:41373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:39377,127.5.61.61:42795,127.5.61.60:42451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:20:28.316736 20701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:28.317051 20701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:28.317210 20701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:20:28.324316 20701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:20:28.324441 20701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:28.324540 20701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:28.324638 20701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:20:28.324720 20701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:20:28.334138 20701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:41373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:39377,127.5.61.61:42795,127.5.61.60:42451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:39377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:28.335402 20701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:28.336735 20701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:28.348304 20708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:28.348421 20707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:28.348395 20710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:28.349331 20701 server_base.cc:1034] running on GCE node
I20260809 20:20:28.350034 20701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:28.351291 20701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:28.352519 20701 hybrid_clock.cc:648] HybridClock initialized: now 1786306828352479 us; error 51 us; skew 500 ppm
I20260809 20:20:28.352841 20701 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:20:28 dist-test-slave-6k22 krb5kdc[20684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306828, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:28.361070 20701 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:20:28.362567 20701 webserver.cc:460] Webserver started at http://127.5.61.62:36789/ using document root <none> and password file <none>
I20260809 20:20:28.363286 20701 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:28.363435 20701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:20:28.363801 20701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:20:28.366890 20701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "31915f64efb545bfa55afb28a2eb2563"
format_stamp: "Formatted at 2026-08-09 20:20:28 on dist-test-slave-6k22"
server_key: "47b1a700a63435a2e46f23a02af1b8cf"
server_key_iv: "f16ced9cf366557b29502dd97c0d8506"
server_key_version: "encryptionkey@0"
I20260809 20:20:28.367772 20701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "31915f64efb545bfa55afb28a2eb2563"
format_stamp: "Formatted at 2026-08-09 20:20:28 on dist-test-slave-6k22"
server_key: "47b1a700a63435a2e46f23a02af1b8cf"
server_key_iv: "f16ced9cf366557b29502dd97c0d8506"
server_key_version: "encryptionkey@0"
I20260809 20:20:28.372781 20701 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 20:20:28.376212 20717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:28.377557 20701 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:20:28.377736 20701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "31915f64efb545bfa55afb28a2eb2563"
format_stamp: "Formatted at 2026-08-09 20:20:28 on dist-test-slave-6k22"
server_key: "47b1a700a63435a2e46f23a02af1b8cf"
server_key_iv: "f16ced9cf366557b29502dd97c0d8506"
server_key_version: "encryptionkey@0"
I20260809 20:20:28.377925 20701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:28.406790 20701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:28.410655 20701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:28.410941 20701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:28.429701 20701 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:39377
I20260809 20:20:28.429744 20779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:39377 every 8 connection(s)
I20260809 20:20:28.431051 20701 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:20:28.434785 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 20701
I20260809 20:20:28.434877 20780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:20:28.435045 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:20:28.435953 5364 external_mini_cluster.cc:1428] Setting key 6d9b8d2a8c1e1f88ce45098a00db92e5
I20260809 20:20:28.445013 20780 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39377 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 20:20:28 dist-test-slave-6k22 krb5kdc[20684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306828, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.61.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.61@KRBTEST.COM" created.
I20260809 20:20:28.468356 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:28.448949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:39377 (local address 127.0.0.1:49160)
0809 20:20:28.450076 (+ 1127us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:28.450109 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:28.451882 (+ 1773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:28.452685 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:28.452700 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:28.453377 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:20:28.453689 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:28.453704 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:28.455215 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:28.455220 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:28.456370 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:28.456382 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:28.457282 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:28.457289 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:28.457527 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:28.458185 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:20:28.458207 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:20:28.460336 (+ 2129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:20:28.463817 (+ 3481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:28.463829 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:28.463840 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:28.464348 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:28.464853 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:28.464861 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:28.464864 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:28.465065 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:28.465870 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:20:28.465880 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:20:28.466085 (+ 205us) client_negotiation.cc:769] Sending connection context
0809 20:20:28.466374 (+ 289us) client_negotiation.cc:240] Negotiation successful
0809 20:20:28.466953 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":939,"mutex_wait_us":136,"thread_start_us":736,"threads_started":1}
I20260809 20:20:28.468390 20783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:28.448946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49160 (local address 127.5.61.62:39377)
0809 20:20:28.449771 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:28.449787 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:28.450231 (+ 444us) server_negotiation.cc:407] Connection header received
0809 20:20:28.452226 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:28.452232 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:28.452445 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:28.452677 (+ 232us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:28.453900 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:28.455025 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:28.456507 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:28.457043 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:28.457251 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:28.460552 (+ 3301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:28.460559 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:28.460588 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:28.460615 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:28.463691 (+ 3076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:28.464487 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:28.464495 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:28.464500 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:28.464739 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:28.465197 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:28.465202 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:28.465206 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:28.465703 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:28.465933 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:28.466517 (+ 584us) server_negotiation.cc:299] Negotiation successful
0809 20:20:28.466835 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":593,"thread_start_us":377,"threads_started":1}
I20260809 20:20:28.471683 20780 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } has no permanent_uuid. Determining permanent_uuid...
I20260809 20:20:28.472703 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:28.472298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:42795 (local address 127.0.0.1:38990)
0809 20:20:28.472463 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:28.472579 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:42795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 20:20:28.472940 20719 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.61.61:42795: connect: Connection refused (error 111)
I20260809 20:20:28.474560 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:28.474207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:42795 (local address 127.0.0.1:38998)
0809 20:20:28.474353 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:28.474439 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:42795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 20:20:28.474977 20780 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.61:42795: Network error: Client connection negotiation failed: client connection to 127.5.61.61:42795: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.61.keytab.
Entry for principal kudu/127.5.61.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.61.keytab.
I20260809 20:20:28.495095 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.61:42795
--webserver_interface=127.5.61.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.61.keytab
--principal=kudu/127.5.61.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:41373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:39377,127.5.61.61:42795,127.5.61.60:42451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 20:20:28.524437 20780 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } attempt: 1
I20260809 20:20:28.525759 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:28.525195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:42795 (local address 127.0.0.1:39000)
0809 20:20:28.525439 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:28.525577 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:42795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 20:20:28.526944 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:28.526450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:42795 (local address 127.0.0.1:39006)
0809 20:20:28.526641 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:28.526735 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:42795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260809 20:20:28.527155 20780 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.61:42795: Network error: Client connection negotiation failed: client connection to 127.5.61.61:42795: connect: Connection refused (error 111)
I20260809 20:20:28.595448 20780 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } attempt: 2
I20260809 20:20:28.596879 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:28.596277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:42795 (local address 127.0.0.1:39020)
0809 20:20:28.596555 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:28.596694 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:42795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260809 20:20:28.597988 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:28.597557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:42795 (local address 127.0.0.1:39034)
0809 20:20:28.597726 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:28.597813 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:42795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 20:20:28.598194 20780 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.61:42795: Network error: Client connection negotiation failed: client connection to 127.5.61.61:42795: connect: Connection refused (error 111)
I20260809 20:20:28.689482 20780 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } attempt: 3
I20260809 20:20:28.690896 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:28.690239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:42795 (local address 127.0.0.1:39044)
0809 20:20:28.690520 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:28.690689 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:42795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260809 20:20:28.692291 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:28.691759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:42795 (local address 127.0.0.1:39052)
0809 20:20:28.691955 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:28.692063 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:42795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 20:20:28.692589 20780 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.61:42795: Network error: Client connection negotiation failed: client connection to 127.5.61.61:42795: connect: Connection refused (error 111)
W20260809 20:20:28.833876 20787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:28.834241 20787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:28.834420 20787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 20:20:28.835865 20780 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } attempt: 4
I20260809 20:20:28.837250 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:28.836585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:42795 (local address 127.0.0.1:39064)
0809 20:20:28.836947 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:28.837059 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:42795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
I20260809 20:20:28.838413 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:28.837931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:42795 (local address 127.0.0.1:39080)
0809 20:20:28.838100 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:28.838243 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:42795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 20:20:28.838627 20780 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.61:42795: Network error: Client connection negotiation failed: client connection to 127.5.61.61:42795: connect: Connection refused (error 111)
W20260809 20:20:28.843374 20787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:20:28.843510 20787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:28.843653 20787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:28.843747 20787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:20:28.843828 20787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:20:28.854583 20787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:41373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:39377,127.5.61.61:42795,127.5.61.60:42451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.61.keytab
--principal=kudu/127.5.61.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.61:42795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.61.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:28.855997 20787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:28.857383 20787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:28.869800 20793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:28.869972 20794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:28.870036 20796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:28.871073 20787 server_base.cc:1034] running on GCE node
I20260809 20:20:28.871901 20787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:28.873157 20787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:28.874356 20787 hybrid_clock.cc:648] HybridClock initialized: now 1786306828874313 us; error 28 us; skew 500 ppm
I20260809 20:20:28.874691 20787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:20:28 dist-test-slave-6k22 krb5kdc[20684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306828, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:28.877882 20787 init.cc:377] Logged in from keytab as kudu/127.5.61.61@KRBTEST.COM (short username kudu)
I20260809 20:20:28.879310 20787 webserver.cc:460] Webserver started at http://127.5.61.61:46289/ using document root <none> and password file <none>
I20260809 20:20:28.880054 20787 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:28.880198 20787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:20:28.880522 20787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:20:28.883507 20787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-1/data/instance:
uuid: "44cd577cb90142b8a519b60c5e7f7758"
format_stamp: "Formatted at 2026-08-09 20:20:28 on dist-test-slave-6k22"
server_key: "d66962a86c990485ae1e2a7041bf0cfa"
server_key_iv: "b49aadbedba2062e9fa79fa4b650314a"
server_key_version: "encryptionkey@0"
I20260809 20:20:28.884353 20787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-1/wal/instance:
uuid: "44cd577cb90142b8a519b60c5e7f7758"
format_stamp: "Formatted at 2026-08-09 20:20:28 on dist-test-slave-6k22"
server_key: "d66962a86c990485ae1e2a7041bf0cfa"
server_key_iv: "b49aadbedba2062e9fa79fa4b650314a"
server_key_version: "encryptionkey@0"
I20260809 20:20:28.889385 20787 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 20:20:28.893260 20803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:28.894534 20787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 20:20:28.894701 20787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-1/wal
uuid: "44cd577cb90142b8a519b60c5e7f7758"
format_stamp: "Formatted at 2026-08-09 20:20:28 on dist-test-slave-6k22"
server_key: "d66962a86c990485ae1e2a7041bf0cfa"
server_key_iv: "b49aadbedba2062e9fa79fa4b650314a"
server_key_version: "encryptionkey@0"
I20260809 20:20:28.894874 20787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:28.922726 20787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:28.926882 20787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:28.927177 20787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:28.945454 20787 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.61:42795
I20260809 20:20:28.945487 20863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.61:42795 every 8 connection(s)
I20260809 20:20:28.946792 20787 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-1/data/info.pb
I20260809 20:20:28.949513 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 20787
I20260809 20:20:28.949704 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-1/wal/instance
I20260809 20:20:28.950169 5364 external_mini_cluster.cc:1428] Setting key fc43488246b32eaf8434005a6b9526d0
I20260809 20:20:28.950453 20864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:20:28.956758 20864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39377 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 20:20:28 dist-test-slave-6k22 krb5kdc[20684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306828, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.61@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.61.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.60@KRBTEST.COM" created.
I20260809 20:20:28.978338 20783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:28.959335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49164 (local address 127.5.61.62:39377)
0809 20:20:28.959510 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:28.959514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:28.960925 (+ 1411us) server_negotiation.cc:407] Connection header received
0809 20:20:28.962743 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:28.962753 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:28.962850 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:28.963012 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:28.964619 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:28.965767 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:28.967254 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:28.968028 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:28.968166 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:28.971490 (+ 3324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:28.971498 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:28.971504 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:28.971547 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:28.975231 (+ 3684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:28.976079 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:28.976083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:28.976085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:28.976205 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:28.976673 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:28.976676 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:28.976678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:28.976976 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:28.977086 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:28.977807 (+ 721us) server_negotiation.cc:299] Negotiation successful
0809 20:20:28.977991 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 20:20:28.979506 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:28.959906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:39377 (local address 127.0.0.1:49164)
0809 20:20:28.960738 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:28.960792 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:28.962467 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:28.963206 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:28.963213 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:28.963892 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:20:28.964443 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:28.964460 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:28.965911 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:28.965915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:28.967104 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:28.967115 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:28.968216 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:28.968221 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:28.968409 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:28.969071 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:20:28.969091 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:20:28.971255 (+ 2164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:20:28.975388 (+ 4133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:28.975395 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:28.975407 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:28.975962 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:28.976304 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:28.976307 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:28.976309 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:28.976501 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:28.977083 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:20:28.977088 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:20:28.977402 (+ 314us) client_negotiation.cc:769] Sending connection context
0809 20:20:28.977638 (+ 236us) client_negotiation.cc:240] Negotiation successful
0809 20:20:28.978018 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":366,"threads_started":1}
I20260809 20:20:28.981375 20864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 20:20:28 dist-test-slave-6k22 krb5kdc[20684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306828, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.61@KRBTEST.COM for kudu/127.5.61.61@KRBTEST.COM
I20260809 20:20:28.995193 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:28.982011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:42795 (local address 127.0.0.1:39096)
0809 20:20:28.982173 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:28.982188 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:28.982300 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:28.983171 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:28.983174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:28.983628 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:20:28.983761 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:28.983769 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:28.985036 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:28.985042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:28.985879 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:28.985889 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:28.986687 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:28.986692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:28.986900 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:28.987482 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:20:28.987499 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:20:28.989794 (+ 2295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:20:28.992763 (+ 2969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:28.992768 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:28.992773 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:28.993117 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:28.993565 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:28.993571 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:28.993574 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:28.993716 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:28.994281 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:20:28.994286 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:20:28.994447 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 20:20:28.994582 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 20:20:28.994842 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 20:20:28.995334 20869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:28.982281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39096 (local address 127.5.61.61:42795)
0809 20:20:28.982813 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:28.982818 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:28.982842 (+ 24us) server_negotiation.cc:407] Connection header received
0809 20:20:28.982880 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:28.982885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:28.983034 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:28.983178 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:28.983899 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:28.984890 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:28.986021 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:28.986545 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:28.986681 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:28.989989 (+ 3308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:28.989999 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:28.990011 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:28.990042 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:28.992577 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:28.993261 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:28.993267 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:28.993269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:28.993450 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:28.993832 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:28.993839 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:28.993842 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:28.994154 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:28.994392 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:28.994717 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 20:20:28.994944 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":403,"thread_start_us":282,"threads_started":1}
I20260809 20:20:28.997972 20864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } has no permanent_uuid. Determining permanent_uuid...
I20260809 20:20:28.999392 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:28.998828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42451 (local address 127.0.0.1:53292)
0809 20:20:28.998984 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:28.999182 (+ 198us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 20:20:28.999850 20806 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
I20260809 20:20:29.001492 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.001149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42451 (local address 127.0.0.1:53300)
0809 20:20:29.001317 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:29.001396 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 20:20:29.001715 20864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:42451: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.60.keytab.
Entry for principal kudu/127.5.61.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.60.keytab.
I20260809 20:20:29.012893 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.60:42451
--webserver_interface=127.5.61.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.60.keytab
--principal=kudu/127.5.61.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:41373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:39377,127.5.61.61:42795,127.5.61.60:42451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 20:20:29.051070 20864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } attempt: 1
I20260809 20:20:29.052443 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.051848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42451 (local address 127.0.0.1:53312)
0809 20:20:29.052103 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:29.052254 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 20:20:29.053462 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.053086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42451 (local address 127.0.0.1:53322)
0809 20:20:29.053253 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:29.053333 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 20:20:29.053683 20864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:42451: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
I20260809 20:20:29.122030 20864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } attempt: 2
I20260809 20:20:29.123292 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.122867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42451 (local address 127.0.0.1:53324)
0809 20:20:29.123053 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:29.123140 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 20:20:29.124321 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.124039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42451 (local address 127.0.0.1:53328)
0809 20:20:29.124159 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:29.124221 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 20:20:29.124590 20864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:42451: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
I20260809 20:20:29.138011 20780 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } attempt: 5
Aug 09 20:20:29 dist-test-slave-6k22 krb5kdc[20684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306828, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for kudu/127.5.61.61@KRBTEST.COM
I20260809 20:20:29.151161 20869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.138852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39100 (local address 127.5.61.61:42795)
0809 20:20:29.139033 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:29.139041 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:29.139062 (+ 21us) server_negotiation.cc:407] Connection header received
0809 20:20:29.139127 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:29.139132 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:29.139203 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:29.139310 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:29.140163 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.140838 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.141981 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.142432 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.142564 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:29.145906 (+ 3342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:29.145910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:29.145913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:29.145943 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:29.148618 (+ 2675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:29.149253 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:29.149260 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:29.149268 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:29.149380 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:29.149790 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:29.149796 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:29.149801 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:29.150071 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:29.150244 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:29.150601 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 20:20:29.150839 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 20:20:29.151172 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.138660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:42795 (local address 127.0.0.1:39100)
0809 20:20:29.138824 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:29.138845 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:29.138988 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:29.139352 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:29.139359 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:29.139840 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:20:29.140016 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:29.140025 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.141007 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.141018 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:29.141832 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:29.141843 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.142574 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.142579 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:29.142793 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:29.143401 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:20:29.143418 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:20:29.145675 (+ 2257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:20:29.148771 (+ 3096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:29.148776 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:29.148780 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:29.149120 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:29.149512 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:29.149517 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:29.149521 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:29.149659 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:29.150204 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:20:29.150209 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:20:29.150360 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 20:20:29.150502 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 20:20:29.150838 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 20:20:29.152315 20780 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } has no permanent_uuid. Determining permanent_uuid...
I20260809 20:20:29.153299 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.152913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42451 (local address 127.0.0.1:53330)
0809 20:20:29.153085 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:29.153162 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 20:20:29.154260 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.153939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42451 (local address 127.0.0.1:53338)
0809 20:20:29.154092 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:29.154162 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 20:20:29.154513 20780 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:42451: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
I20260809 20:20:29.205761 20780 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } attempt: 1
I20260809 20:20:29.206982 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.206526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42451 (local address 127.0.0.1:53354)
0809 20:20:29.206723 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:29.206818 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 20:20:29.208022 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.207696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42451 (local address 127.0.0.1:53360)
0809 20:20:29.207851 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:29.207928 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 20:20:29.208246 20780 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:42451: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
I20260809 20:20:29.215811 20864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } attempt: 3
I20260809 20:20:29.216789 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.216366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42451 (local address 127.0.0.1:53364)
0809 20:20:29.216565 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:29.216642 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 20:20:29.217813 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.217395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42451 (local address 127.0.0.1:53372)
0809 20:20:29.217631 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:29.217700 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260809 20:20:29.218019 20864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:42451: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
I20260809 20:20:29.276472 20780 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } attempt: 2
I20260809 20:20:29.277621 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.277115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42451 (local address 127.0.0.1:53384)
0809 20:20:29.277348 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:29.277443 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 20:20:29.278573 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.278227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42451 (local address 127.0.0.1:53400)
0809 20:20:29.278385 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:29.278479 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 20:20:29.278770 20780 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:42451: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
W20260809 20:20:29.349160 20873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:29.349462 20873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:29.349607 20873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:20:29.356571 20873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:20:29.356699 20873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:29.356781 20873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:29.356884 20873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:20:29.356959 20873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:20:29.361222 20864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } attempt: 4
I20260809 20:20:29.362268 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.361812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42451 (local address 127.0.0.1:53408)
0809 20:20:29.362044 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:29.362124 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 20:20:29.363224 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.362928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42451 (local address 127.0.0.1:53416)
0809 20:20:29.363065 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:29.363131 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 20:20:29.363435 20864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:42451: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
I20260809 20:20:29.366906 20873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:41373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:39377,127.5.61.61:42795,127.5.61.60:42451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.60.keytab
--principal=kudu/127.5.61.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.60:42451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.61.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:29.368232 20873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:29.369500 20873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:29.381040 20878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:29.381134 20880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:29.381168 20882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:29.381999 20873 server_base.cc:1034] running on GCE node
I20260809 20:20:29.382694 20873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:29.384222 20873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:29.384967 20780 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } attempt: 3
I20260809 20:20:29.385422 20873 hybrid_clock.cc:648] HybridClock initialized: now 1786306829385382 us; error 29 us; skew 500 ppm
I20260809 20:20:29.385767 20873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:20:29.386024 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.385524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42451 (local address 127.0.0.1:53432)
0809 20:20:29.385743 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:29.385828 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 20:20:29.387014 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.386650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42451 (local address 127.0.0.1:53444)
0809 20:20:29.386831 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:29.386915 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260809 20:20:29.387249 20780 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:42451: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42451: connect: Connection refused (error 111)
Aug 09 20:20:29 dist-test-slave-6k22 krb5kdc[20684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306829, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:29.389181 20873 init.cc:377] Logged in from keytab as kudu/127.5.61.60@KRBTEST.COM (short username kudu)
I20260809 20:20:29.390659 20873 webserver.cc:460] Webserver started at http://127.5.61.60:34775/ using document root <none> and password file <none>
I20260809 20:20:29.391328 20873 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:29.391465 20873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:20:29.391799 20873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:20:29.394847 20873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-2/data/instance:
uuid: "3c26257d5b55455d856f17a6209f48b3"
format_stamp: "Formatted at 2026-08-09 20:20:29 on dist-test-slave-6k22"
server_key: "03d71943d57776cc9778d567300edd0f"
server_key_iv: "464b64bc91b9dada6dc228f139d072f4"
server_key_version: "encryptionkey@0"
I20260809 20:20:29.395645 20873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-2/wal/instance:
uuid: "3c26257d5b55455d856f17a6209f48b3"
format_stamp: "Formatted at 2026-08-09 20:20:29 on dist-test-slave-6k22"
server_key: "03d71943d57776cc9778d567300edd0f"
server_key_iv: "464b64bc91b9dada6dc228f139d072f4"
server_key_version: "encryptionkey@0"
I20260809 20:20:29.400826 20873 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 20:20:29.404239 20891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:29.405601 20873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:20:29.405781 20873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-2/wal
uuid: "3c26257d5b55455d856f17a6209f48b3"
format_stamp: "Formatted at 2026-08-09 20:20:29 on dist-test-slave-6k22"
server_key: "03d71943d57776cc9778d567300edd0f"
server_key_iv: "464b64bc91b9dada6dc228f139d072f4"
server_key_version: "encryptionkey@0"
I20260809 20:20:29.405967 20873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:29.432940 20873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:29.437413 20873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:29.437728 20873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:29.456600 20873 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.60:42451
I20260809 20:20:29.456645 20950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.60:42451 every 8 connection(s)
I20260809 20:20:29.457927 20873 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-2/data/info.pb
I20260809 20:20:29.461622 20951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:20:29.467803 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 20873
I20260809 20:20:29.468022 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786306623011809-5364-0/minicluster-data/master-2/wal/instance
I20260809 20:20:29.468681 5364 external_mini_cluster.cc:1428] Setting key 29fd3369ff5d5ce6bd52ff4d1a24f725
I20260809 20:20:29.468693 20951 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39377 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 20:20:29 dist-test-slave-6k22 krb5kdc[20684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306829, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.60@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
Aug 09 20:20:29 dist-test-slave-6k22 krb5kdc[20684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.60@KRBTEST.COM
I20260809 20:20:29.488343 20783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.471837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49166 (local address 127.5.61.62:39377)
0809 20:20:29.472045 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:29.472052 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:29.472781 (+ 729us) server_negotiation.cc:407] Connection header received
0809 20:20:29.474013 (+ 1232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:29.474020 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:29.474100 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:29.474208 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:29.475451 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.476262 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.477805 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.478248 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.478399 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:29.482194 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:29.482201 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:29.482206 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:29.482258 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:29.485133 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:29.485904 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:29.485911 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:29.485920 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:29.486047 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:29.486510 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:29.486515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:29.486519 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:29.486841 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:29.487015 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:29.487741 (+ 726us) server_negotiation.cc:299] Negotiation successful
0809 20:20:29.487980 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 20:20:29.489575 20955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.471738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:39377 (local address 127.0.0.1:49166)
0809 20:20:29.472614 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:29.472668 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:29.473781 (+ 1113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:29.474270 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:29.474276 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:29.474901 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:20:29.475277 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:29.475294 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.476456 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.476459 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:29.477665 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:29.477678 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.478531 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.478539 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:29.478811 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:29.479519 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:20:29.479541 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:20:29.481937 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:20:29.485272 (+ 3335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:29.485283 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:29.485293 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:29.485772 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:29.486160 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:29.486165 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:29.486168 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:29.486355 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:29.486979 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:20:29.486985 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:20:29.487292 (+ 307us) client_negotiation.cc:769] Sending connection context
0809 20:20:29.487626 (+ 334us) client_negotiation.cc:240] Negotiation successful
0809 20:20:29.488045 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"mutex_wait_us":36,"thread_start_us":495,"threads_started":1}
I20260809 20:20:29.489935 20954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.470634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53452 (local address 127.5.61.60:42451)
0809 20:20:29.471460 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:29.471474 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:29.471522 (+ 48us) server_negotiation.cc:407] Connection header received
0809 20:20:29.473769 (+ 2247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:29.473789 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:29.474010 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:29.474211 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:29.475723 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.476933 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.478350 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.478988 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.479129 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:29.483745 (+ 4616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:29.483754 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:29.483765 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:29.483800 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:29.486658 (+ 2858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:29.487477 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:29.487485 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:29.487487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:29.487665 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:29.488118 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:29.488123 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:29.488126 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:29.488559 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:29.488870 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:29.489198 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 20:20:29.489450 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":612,"thread_start_us":382,"threads_started":1}
I20260809 20:20:29.491393 20951 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 20:20:29 dist-test-slave-6k22 krb5kdc[20684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306829, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.60@KRBTEST.COM for kudu/127.5.61.61@KRBTEST.COM
Aug 09 20:20:29 dist-test-slave-6k22 krb5kdc[20684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:20:29.505813 20955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.492089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:42795 (local address 127.0.0.1:39104)
0809 20:20:29.492332 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:29.492347 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:29.492461 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:29.492876 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:29.492881 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:29.493248 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:20:29.493391 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:29.493399 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.494396 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.494403 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:29.495828 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:29.495841 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.496821 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.496824 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:29.497000 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:29.497594 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:20:29.497615 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:20:29.499983 (+ 2368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:20:29.503436 (+ 3453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:29.503443 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:29.503446 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:29.503991 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:29.504415 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:29.504418 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:29.504420 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:29.504554 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:29.505166 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:20:29.505169 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:20:29.505296 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 20:20:29.505391 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 20:20:29.505582 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132}
I20260809 20:20:29.506139 20869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.492331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39104 (local address 127.5.61.61:42795)
0809 20:20:29.492544 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:29.492550 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:29.492574 (+ 24us) server_negotiation.cc:407] Connection header received
0809 20:20:29.492644 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:29.492649 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:29.492730 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:29.492846 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:29.493554 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.494238 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.495985 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.496681 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.496819 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:29.500203 (+ 3384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:29.500212 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:29.500218 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:29.500283 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:29.503311 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:29.504119 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:29.504126 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:29.504130 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:29.504295 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:29.504668 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:29.504673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:29.504677 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:29.505040 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:29.505190 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:29.505560 (+ 370us) server_negotiation.cc:299] Negotiation successful
0809 20:20:29.505772 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 20:20:29.506978 20951 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } has no permanent_uuid. Determining permanent_uuid...
I20260809 20:20:29.509331 20783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.492849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49176 (local address 127.5.61.62:39377)
0809 20:20:29.492992 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:29.492998 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:29.493084 (+ 86us) server_negotiation.cc:407] Connection header received
0809 20:20:29.493253 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:29.493258 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:29.493325 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:29.493420 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:29.494670 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.495826 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.497608 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.498250 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.498395 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:29.503250 (+ 4855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:29.503256 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:29.503262 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:29.503298 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:29.506563 (+ 3265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:29.507217 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:29.507223 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:29.507226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:29.507377 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:29.507873 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:29.507880 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:29.507884 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:29.508171 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:29.508468 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:29.508782 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 20:20:29.508994 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 20:20:29 dist-test-slave-6k22 krb5kdc[20684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306829, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.60@KRBTEST.COM for kudu/127.5.61.60@KRBTEST.COM
Aug 09 20:20:29 dist-test-slave-6k22 krb5kdc[20684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.61@KRBTEST.COM
I20260809 20:20:29.521023 20955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.507741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42451 (local address 127.0.0.1:53468)
0809 20:20:29.507902 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:29.507921 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:29.508015 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:29.508282 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:29.508286 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:29.508644 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:20:29.508771 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:29.508779 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.510135 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.510146 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:29.511514 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:29.511530 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.512238 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.512243 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:29.512502 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:29.513039 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:20:29.513060 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:20:29.515192 (+ 2132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:20:29.518142 (+ 2950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:29.518148 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:29.518151 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:29.518523 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:29.519391 (+ 868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:29.519394 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:29.519396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:29.519516 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:29.520056 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:20:29.520061 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:20:29.520186 (+ 125us) client_negotiation.cc:769] Sending connection context
0809 20:20:29.520328 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 20:20:29.520621 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":32}
I20260809 20:20:29.521047 20954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.507901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53468 (local address 127.5.61.60:42451)
0809 20:20:29.508050 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:29.508057 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:29.508075 (+ 18us) server_negotiation.cc:407] Connection header received
0809 20:20:29.508114 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:29.508118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:29.508171 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:29.508265 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:29.508974 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.509933 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.511675 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.512101 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.512231 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:29.515374 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:29.515379 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:29.515384 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:29.515426 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:29.518012 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:29.518637 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:29.518644 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:29.518647 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:29.518765 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:29.519662 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:29.519669 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:29.519672 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:29.519940 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:29.520091 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:29.520431 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 20:20:29.520751 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":131,"server-negotiator.queue_time_us":58}
I20260809 20:20:29.525938 20869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.511000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39114 (local address 127.5.61.61:42795)
0809 20:20:29.511142 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:29.511147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:29.511164 (+ 17us) server_negotiation.cc:407] Connection header received
0809 20:20:29.511265 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:29.511270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:29.511333 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:29.511433 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:29.512679 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.513818 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.515129 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.515611 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.515743 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:29.520249 (+ 4506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:29.520254 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:29.520256 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:29.520285 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:29.523270 (+ 2985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:29.523989 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:29.523996 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:29.523999 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:29.524140 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:29.524545 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:29.524549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:29.524553 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:29.524821 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:29.525119 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:29.525405 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 20:20:29.525625 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 20:20:29.526361 20951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c26257d5b55455d856f17a6209f48b3: Bootstrap starting.
I20260809 20:20:29.530254 20951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c26257d5b55455d856f17a6209f48b3: Neither blocks nor log segments found. Creating new log.
I20260809 20:20:29.530692 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:20:29.531418 20951 log.cc:824] T 00000000000000000000000000000000 P 3c26257d5b55455d856f17a6209f48b3: Log is configured to *not* fsync() on all Append() calls
I20260809 20:20:29.534300 20951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c26257d5b55455d856f17a6209f48b3: No bootstrap required, opened a new log
I20260809 20:20:29.546291 20951 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c26257d5b55455d856f17a6209f48b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31915f64efb545bfa55afb28a2eb2563" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39377 } } peers { permanent_uuid: "44cd577cb90142b8a519b60c5e7f7758" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } } peers { permanent_uuid: "3c26257d5b55455d856f17a6209f48b3" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } }
I20260809 20:20:29.546742 20951 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c26257d5b55455d856f17a6209f48b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:20:29.546883 20951 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c26257d5b55455d856f17a6209f48b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c26257d5b55455d856f17a6209f48b3, State: Initialized, Role: FOLLOWER
I20260809 20:20:29.547701 20951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c26257d5b55455d856f17a6209f48b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31915f64efb545bfa55afb28a2eb2563" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39377 } } peers { permanent_uuid: "44cd577cb90142b8a519b60c5e7f7758" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } } peers { permanent_uuid: "3c26257d5b55455d856f17a6209f48b3" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } }
I20260809 20:20:29.554673 20964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c26257d5b55455d856f17a6209f48b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31915f64efb545bfa55afb28a2eb2563" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39377 } } peers { permanent_uuid: "44cd577cb90142b8a519b60c5e7f7758" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } } peers { permanent_uuid: "3c26257d5b55455d856f17a6209f48b3" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } } }
I20260809 20:20:29.555140 20964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c26257d5b55455d856f17a6209f48b3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:20:29.556034 20783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.536028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49178 (local address 127.5.61.62:39377)
0809 20:20:29.536206 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:29.536212 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:29.536864 (+ 652us) server_negotiation.cc:407] Connection header received
0809 20:20:29.537181 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:29.537189 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:29.537293 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:29.537396 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:29.540585 (+ 3189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.541333 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.543807 (+ 2474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.544518 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.544616 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:29.548545 (+ 3929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:29.548551 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:29.548556 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:29.548595 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:29.552381 (+ 3786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:29.553409 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:29.553415 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:29.553420 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:29.553606 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:29.554118 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:29.554123 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:29.554126 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:29.554510 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:29.554761 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:29.555384 (+ 623us) server_negotiation.cc:299] Negotiation successful
0809 20:20:29.555652 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 20:20:29.557157 20951 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c26257d5b55455d856f17a6209f48b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:20:29.560249 20869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.537025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39118 (local address 127.5.61.61:42795)
0809 20:20:29.537204 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:29.537212 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:29.541676 (+ 4464us) server_negotiation.cc:407] Connection header received
0809 20:20:29.541828 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:29.541834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:29.541924 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:29.542025 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:29.544334 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.545412 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.548377 (+ 2965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.549032 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.549137 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:29.553197 (+ 4060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:29.553203 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:29.553208 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:29.553246 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:29.556803 (+ 3557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:29.557709 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:29.557715 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:29.557718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:29.557922 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:29.558399 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:29.558407 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:29.558409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:29.558770 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:29.559050 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:29.559612 (+ 562us) server_negotiation.cc:299] Negotiation successful
0809 20:20:29.559842 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 20:20:29.563189 20954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.536593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53474 (local address 127.5.61.60:42451)
0809 20:20:29.536750 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:29.536757 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:29.540430 (+ 3673us) server_negotiation.cc:407] Connection header received
0809 20:20:29.542058 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:29.542063 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:29.542133 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:29.542206 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:29.544657 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.545752 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.549003 (+ 3251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.550014 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.550144 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:29.554836 (+ 4692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:29.554848 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:29.554852 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:29.554897 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:29.559236 (+ 4339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:29.561068 (+ 1832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:29.561077 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:29.561083 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:29.561337 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:29.561868 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:29.561873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:29.561875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:29.562163 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:29.562345 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:29.562770 (+ 425us) server_negotiation.cc:299] Negotiation successful
0809 20:20:29.562924 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 20:20:29.564450 20780 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } attempt: 4
Aug 09 20:20:29 dist-test-slave-6k22 krb5kdc[20684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306828, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for kudu/127.5.61.60@KRBTEST.COM
W20260809 20:20:29.577500 20976 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3c26257d5b55455d856f17a6209f48b3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 20:20:29.577884 20976 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 20:20:29.585391 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.565005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42451 (local address 127.0.0.1:53490)
0809 20:20:29.565248 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:29.565273 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:29.565414 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:29.565747 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:29.565755 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:29.566417 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:20:29.566697 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:29.566714 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.568459 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.568463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:29.569756 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:29.569774 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.571105 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.571112 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:29.571258 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:29.572109 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:20:29.572134 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:20:29.574905 (+ 2771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:20:29.581579 (+ 6674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:29.581585 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:29.581589 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:29.582134 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:29.582605 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:29.582609 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:29.582612 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:29.582806 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:29.583851 (+ 1045us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:20:29.583856 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:20:29.584303 (+ 447us) client_negotiation.cc:769] Sending connection context
0809 20:20:29.584604 (+ 301us) client_negotiation.cc:240] Negotiation successful
0809 20:20:29.584979 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 20:20:29.586544 20954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.565077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53490 (local address 127.5.61.60:42451)
0809 20:20:29.565243 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:29.565247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:29.565350 (+ 103us) server_negotiation.cc:407] Connection header received
0809 20:20:29.565524 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:29.565527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:29.565596 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:29.565670 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:29.566941 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.568311 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.569925 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.570743 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.570863 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:29.576087 (+ 5224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:29.576093 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:29.576097 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:29.576125 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:29.581433 (+ 5308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:29.582287 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:29.582293 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:29.582297 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:29.582482 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:29.582945 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:29.582953 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:29.582958 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:29.583489 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:29.585772 (+ 2283us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:29.585894 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 20:20:29.586148 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 20:20:29.572898 20873 master_runner.cc:186] Error getting master registration for 127.5.61.62:39377: OK, instance_id { permanent_uuid: "31915f64efb545bfa55afb28a2eb2563" instance_seqno: 1786306828424403 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 20:20:29.587167 20873 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 20:20:29.591078 20780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563: Bootstrap starting.
I20260809 20:20:29.595289 20780 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563: Neither blocks nor log segments found. Creating new log.
I20260809 20:20:29.596400 20780 log.cc:824] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563: Log is configured to *not* fsync() on all Append() calls
I20260809 20:20:29.599371 20780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563: No bootstrap required, opened a new log
I20260809 20:20:29.606784 20780 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31915f64efb545bfa55afb28a2eb2563" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39377 } } peers { permanent_uuid: "44cd577cb90142b8a519b60c5e7f7758" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } } peers { permanent_uuid: "3c26257d5b55455d856f17a6209f48b3" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } }
I20260809 20:20:29.607086 20780 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:20:29.607234 20780 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31915f64efb545bfa55afb28a2eb2563, State: Initialized, Role: FOLLOWER
I20260809 20:20:29.607834 20780 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31915f64efb545bfa55afb28a2eb2563" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39377 } } peers { permanent_uuid: "44cd577cb90142b8a519b60c5e7f7758" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } } peers { permanent_uuid: "3c26257d5b55455d856f17a6209f48b3" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } }
I20260809 20:20:29.608757 20980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31915f64efb545bfa55afb28a2eb2563" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39377 } } peers { permanent_uuid: "44cd577cb90142b8a519b60c5e7f7758" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } } peers { permanent_uuid: "3c26257d5b55455d856f17a6209f48b3" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } } }
I20260809 20:20:29.609193 20980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:20:29.610185 20780 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 20:20:29.620360 20991 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 20:20:29.620566 20991 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 20:20:29.619619 20701 master_runner.cc:186] Error getting master registration for 127.5.61.61:42795: OK, instance_id { permanent_uuid: "44cd577cb90142b8a519b60c5e7f7758" instance_seqno: 1786306828940217 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 20:20:29.630940 20701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 20:20:29.662652 20864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } attempt: 5
W20260809 20:20:29.664108 20726 tablet.cc:2369] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 20:20:29 dist-test-slave-6k22 krb5kdc[20684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306828, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.61@KRBTEST.COM for kudu/127.5.61.60@KRBTEST.COM
I20260809 20:20:29.681056 20954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.663525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53496 (local address 127.5.61.60:42451)
0809 20:20:29.663749 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:29.663755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:29.663777 (+ 22us) server_negotiation.cc:407] Connection header received
0809 20:20:29.664012 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:29.664018 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:29.664103 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:29.664213 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:29.665807 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.667045 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.669031 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.669712 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.669847 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:29.673998 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:29.674005 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:29.674012 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:29.674054 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:29.677905 (+ 3851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:29.678744 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:29.678750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:29.678753 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:29.678908 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:29.679343 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:29.679349 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:29.679352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:29.679743 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:29.679907 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:29.680370 (+ 463us) server_negotiation.cc:299] Negotiation successful
0809 20:20:29.680634 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260809 20:20:29.681064 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:29.663339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42451 (local address 127.0.0.1:53496)
0809 20:20:29.663677 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:29.663700 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:29.663821 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:29.664511 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:29.664517 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:29.665080 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:20:29.665294 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:29.665307 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.667232 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.667237 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:29.668877 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:29.668893 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:29.669897 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:29.669904 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:29.670103 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:29.670831 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:20:29.670855 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:20:29.673787 (+ 2932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:20:29.678046 (+ 4259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:29.678052 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:29.678057 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:29.678587 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:29.679036 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:29.679042 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:29.679045 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:29.679212 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:29.679875 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:20:29.679881 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:20:29.680043 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 20:20:29.680221 (+ 178us) client_negotiation.cc:240] Negotiation successful
0809 20:20:29.680509 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174}
I20260809 20:20:29.687418 20864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44cd577cb90142b8a519b60c5e7f7758: Bootstrap starting.
W20260809 20:20:29.689817 20899 tablet.cc:2369] T 00000000000000000000000000000000 P 3c26257d5b55455d856f17a6209f48b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:20:29.691519 20864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44cd577cb90142b8a519b60c5e7f7758: Neither blocks nor log segments found. Creating new log.
I20260809 20:20:29.692857 20864 log.cc:824] T 00000000000000000000000000000000 P 44cd577cb90142b8a519b60c5e7f7758: Log is configured to *not* fsync() on all Append() calls
I20260809 20:20:29.696148 20864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44cd577cb90142b8a519b60c5e7f7758: No bootstrap required, opened a new log
I20260809 20:20:29.707254 20864 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44cd577cb90142b8a519b60c5e7f7758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31915f64efb545bfa55afb28a2eb2563" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39377 } } peers { permanent_uuid: "44cd577cb90142b8a519b60c5e7f7758" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } } peers { permanent_uuid: "3c26257d5b55455d856f17a6209f48b3" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } }
I20260809 20:20:29.707661 20864 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44cd577cb90142b8a519b60c5e7f7758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:20:29.707782 20864 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44cd577cb90142b8a519b60c5e7f7758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44cd577cb90142b8a519b60c5e7f7758, State: Initialized, Role: FOLLOWER
I20260809 20:20:29.708413 20864 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44cd577cb90142b8a519b60c5e7f7758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31915f64efb545bfa55afb28a2eb2563" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39377 } } peers { permanent_uuid: "44cd577cb90142b8a519b60c5e7f7758" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } } peers { permanent_uuid: "3c26257d5b55455d856f17a6209f48b3" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } }
I20260809 20:20:29.709488 20994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44cd577cb90142b8a519b60c5e7f7758 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31915f64efb545bfa55afb28a2eb2563" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39377 } } peers { permanent_uuid: "44cd577cb90142b8a519b60c5e7f7758" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } } peers { permanent_uuid: "3c26257d5b55455d856f17a6209f48b3" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } } }
I20260809 20:20:29.710011 20994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44cd577cb90142b8a519b60c5e7f7758 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:20:29.710883 20864 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44cd577cb90142b8a519b60c5e7f7758 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:20:29.732527 20980 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:20:29.732849 20980 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31915f64efb545bfa55afb28a2eb2563" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39377 } } peers { permanent_uuid: "44cd577cb90142b8a519b60c5e7f7758" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } } peers { permanent_uuid: "3c26257d5b55455d856f17a6209f48b3" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } }
I20260809 20:20:29.734354 20980 leader_election.cc:288] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44cd577cb90142b8a519b60c5e7f7758 (127.5.61.61:42795), 3c26257d5b55455d856f17a6209f48b3 (127.5.61.60:42451)
I20260809 20:20:29.735302 20835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "31915f64efb545bfa55afb28a2eb2563" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44cd577cb90142b8a519b60c5e7f7758" is_pre_election: true
I20260809 20:20:29.735913 20835 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 44cd577cb90142b8a519b60c5e7f7758 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31915f64efb545bfa55afb28a2eb2563 in term 0.
I20260809 20:20:29.736960 20787 master_runner.cc:263] No leader master found from master 44cd577cb90142b8a519b60c5e7f7758
I20260809 20:20:29.736953 20719 leader_election.cc:302] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31915f64efb545bfa55afb28a2eb2563, 44cd577cb90142b8a519b60c5e7f7758; no voters:
I20260809 20:20:29.737097 20787 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 20:20:29.736871 20922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "31915f64efb545bfa55afb28a2eb2563" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c26257d5b55455d856f17a6209f48b3" is_pre_election: true
I20260809 20:20:29.737349 20980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:20:29.737517 20980 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:20:29.737514 20922 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3c26257d5b55455d856f17a6209f48b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31915f64efb545bfa55afb28a2eb2563 in term 0.
I20260809 20:20:29.737628 20980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [term 0 FOLLOWER]: Advancing to term 1
W20260809 20:20:29.742154 21007 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 44cd577cb90142b8a519b60c5e7f7758: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 20:20:29.742332 21007 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 20:20:29.742882 20980 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31915f64efb545bfa55afb28a2eb2563" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39377 } } peers { permanent_uuid: "44cd577cb90142b8a519b60c5e7f7758" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } } peers { permanent_uuid: "3c26257d5b55455d856f17a6209f48b3" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } }
I20260809 20:20:29.745981 20835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "31915f64efb545bfa55afb28a2eb2563" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44cd577cb90142b8a519b60c5e7f7758"
I20260809 20:20:29.746213 20835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44cd577cb90142b8a519b60c5e7f7758 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:20:29.749855 20835 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 44cd577cb90142b8a519b60c5e7f7758 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31915f64efb545bfa55afb28a2eb2563 in term 1.
I20260809 20:20:29.750638 20719 leader_election.cc:302] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31915f64efb545bfa55afb28a2eb2563, 44cd577cb90142b8a519b60c5e7f7758; no voters:
I20260809 20:20:29.754765 20980 leader_election.cc:288] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [CANDIDATE]: Term 1 election: Requested vote from peers 44cd577cb90142b8a519b60c5e7f7758 (127.5.61.61:42795), 3c26257d5b55455d856f17a6209f48b3 (127.5.61.60:42451)
I20260809 20:20:29.754735 20922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "31915f64efb545bfa55afb28a2eb2563" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c26257d5b55455d856f17a6209f48b3"
I20260809 20:20:29.754984 20980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:20:29.755003 20922 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c26257d5b55455d856f17a6209f48b3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:20:29.755134 20980 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [term 1 LEADER]: Becoming Leader. State: Replica: 31915f64efb545bfa55afb28a2eb2563, State: Running, Role: LEADER
I20260809 20:20:29.756570 20980 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31915f64efb545bfa55afb28a2eb2563" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39377 } } peers { permanent_uuid: "44cd577cb90142b8a519b60c5e7f7758" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } } peers { permanent_uuid: "3c26257d5b55455d856f17a6209f48b3" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } }
I20260809 20:20:29.759052 20922 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3c26257d5b55455d856f17a6209f48b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31915f64efb545bfa55afb28a2eb2563 in term 1.
I20260809 20:20:29.767436 20980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31915f64efb545bfa55afb28a2eb2563. Latest consensus state: current_term: 1 leader_uuid: "31915f64efb545bfa55afb28a2eb2563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31915f64efb545bfa55afb28a2eb2563" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39377 } } peers { permanent_uuid: "44cd577cb90142b8a519b60c5e7f7758" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } } peers { permanent_uuid: "3c26257d5b55455d856f17a6209f48b3" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } } }
I20260809 20:20:29.767774 20980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [sys.catalog]: This master's current role is: LEADER
I20260809 20:20:29.778739 21011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:20:29.780874 21011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:20:29.804822 20922 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3c26257d5b55455d856f17a6209f48b3 [term 1 FOLLOWER]: Refusing update from remote peer 31915f64efb545bfa55afb28a2eb2563: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:20:29.805784 20980 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c26257d5b55455d856f17a6209f48b3" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:20:29.813956 20835 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 44cd577cb90142b8a519b60c5e7f7758 [term 1 FOLLOWER]: Refusing update from remote peer 31915f64efb545bfa55afb28a2eb2563: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:20:29.815547 20980 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44cd577cb90142b8a519b60c5e7f7758" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:20:29.848791 20994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44cd577cb90142b8a519b60c5e7f7758 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31915f64efb545bfa55afb28a2eb2563. Latest consensus state: current_term: 1 leader_uuid: "31915f64efb545bfa55afb28a2eb2563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31915f64efb545bfa55afb28a2eb2563" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39377 } } peers { permanent_uuid: "44cd577cb90142b8a519b60c5e7f7758" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } } peers { permanent_uuid: "3c26257d5b55455d856f17a6209f48b3" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } } }
I20260809 20:20:29.849097 20994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44cd577cb90142b8a519b60c5e7f7758 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:20:29.850804 21012 mvcc.cc:205] Tried to move back new op lower bound from 7316712774868881408 to 7316712774693986304. Current Snapshot: MvccSnapshot[applied={T|T < 7316712774868881408}]
I20260809 20:20:29.856304 21015 mvcc.cc:205] Tried to move back new op lower bound from 7316712774868881408 to 7316712774693986304. Current Snapshot: MvccSnapshot[applied={T|T < 7316712774868881408}]
I20260809 20:20:29.860064 20994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44cd577cb90142b8a519b60c5e7f7758 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31915f64efb545bfa55afb28a2eb2563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31915f64efb545bfa55afb28a2eb2563" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39377 } } peers { permanent_uuid: "44cd577cb90142b8a519b60c5e7f7758" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } } peers { permanent_uuid: "3c26257d5b55455d856f17a6209f48b3" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } } }
I20260809 20:20:29.860312 20994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44cd577cb90142b8a519b60c5e7f7758 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:20:29.866621 20980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31915f64efb545bfa55afb28a2eb2563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31915f64efb545bfa55afb28a2eb2563" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39377 } } peers { permanent_uuid: "44cd577cb90142b8a519b60c5e7f7758" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } } peers { permanent_uuid: "3c26257d5b55455d856f17a6209f48b3" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } } }
I20260809 20:20:29.866665 20964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c26257d5b55455d856f17a6209f48b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31915f64efb545bfa55afb28a2eb2563. Latest consensus state: current_term: 1 leader_uuid: "31915f64efb545bfa55afb28a2eb2563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31915f64efb545bfa55afb28a2eb2563" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39377 } } peers { permanent_uuid: "44cd577cb90142b8a519b60c5e7f7758" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } } peers { permanent_uuid: "3c26257d5b55455d856f17a6209f48b3" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } } }
I20260809 20:20:29.866946 20980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [sys.catalog]: This master's current role is: LEADER
I20260809 20:20:29.867004 20964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c26257d5b55455d856f17a6209f48b3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:20:29.869719 21009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31915f64efb545bfa55afb28a2eb2563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31915f64efb545bfa55afb28a2eb2563" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39377 } } peers { permanent_uuid: "44cd577cb90142b8a519b60c5e7f7758" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } } peers { permanent_uuid: "3c26257d5b55455d856f17a6209f48b3" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } } }
I20260809 20:20:29.870157 21009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563 [sys.catalog]: This master's current role is: LEADER
I20260809 20:20:29.871428 21011 catalog_manager.cc:1347] Generated new cluster ID: 50ead57c4329400f9c63eb3a76703c3e
I20260809 20:20:29.871531 21011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:20:29.872720 20964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c26257d5b55455d856f17a6209f48b3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31915f64efb545bfa55afb28a2eb2563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31915f64efb545bfa55afb28a2eb2563" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39377 } } peers { permanent_uuid: "44cd577cb90142b8a519b60c5e7f7758" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 42795 } } peers { permanent_uuid: "3c26257d5b55455d856f17a6209f48b3" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42451 } } }
I20260809 20:20:29.872980 20964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c26257d5b55455d856f17a6209f48b3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:20:29.873206 21013 mvcc.cc:205] Tried to move back new op lower bound from 7316712774868881408 to 7316712774693986304. Current Snapshot: MvccSnapshot[applied={T|T < 7316712774868881408}]
I20260809 20:20:29.918566 21011 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:20:29.920696 21011 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:20:29.948150 21011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31915f64efb545bfa55afb28a2eb2563: Generated new TSK 0
I20260809 20:20:29.949098 21011 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:20:30.579980 20976 catalog_manager.cc:1259] Loaded cluster ID: 50ead57c4329400f9c63eb3a76703c3e
I20260809 20:20:30.580076 20976 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3c26257d5b55455d856f17a6209f48b3: loading cluster ID for follower catalog manager: success
I20260809 20:20:30.582521 20976 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3c26257d5b55455d856f17a6209f48b3: acquiring CA information for follower catalog manager: success
I20260809 20:20:30.584264 20976 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3c26257d5b55455d856f17a6209f48b3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 20:20:30.744484 21007 catalog_manager.cc:1259] Loaded cluster ID: 50ead57c4329400f9c63eb3a76703c3e
I20260809 20:20:30.744588 21007 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 44cd577cb90142b8a519b60c5e7f7758: loading cluster ID for follower catalog manager: success
I20260809 20:20:30.747045 21007 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 44cd577cb90142b8a519b60c5e7f7758: acquiring CA information for follower catalog manager: success
I20260809 20:20:30.748809 21007 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 44cd577cb90142b8a519b60c5e7f7758: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 20:20:31.082799 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 20701
I20260809 20:20:31.084219 20775 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:20:31.193979 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 20701
I20260809 20:20:31.219172 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 20787
I20260809 20:20:31.220427 20857 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:20:31.327395 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 20787
I20260809 20:20:31.351090 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 20873
I20260809 20:20:31.352344 20946 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:20:31.459497 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 20873
2026-08-09T20:20:31Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4901 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:20:31 dist-test-slave-6k22 krb5kdc[21032](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:20:31 dist-test-slave-6k22 krb5kdc[21032](info): set up 2 sockets
Aug 09 20:20:31 dist-test-slave-6k22 krb5kdc[21032](info): commencing operation
krb5kdc: starting...
W20260809 20:20:32.626456 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:20:32 dist-test-slave-6k22 krb5kdc[21032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:20:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:20:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:20:32.874974 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:41961
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:38869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:41961
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:20:33.197772 21047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:33.198078 21047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:33.198231 21047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:20:33.205273 21047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:20:33.205401 21047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:33.205488 21047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:33.205605 21047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:20:33.205694 21047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:20:33.214924 21047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:38869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:41961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:41961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:33.216228 21047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:33.217509 21047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:33.229162 21055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:33.229214 21052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:33.229235 21053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:33.230389 21047 server_base.cc:1034] running on GCE node
I20260809 20:20:33.231334 21047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:33.232797 21047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:33.234089 21047 hybrid_clock.cc:648] HybridClock initialized: now 1786306833233995 us; error 74 us; skew 500 ppm
I20260809 20:20:33.234586 21047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:20:33 dist-test-slave-6k22 krb5kdc[21032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306833, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:33.238453 21047 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:20:33.240530 21047 webserver.cc:460] Webserver started at http://127.5.61.62:35389/ using document root <none> and password file <none>
I20260809 20:20:33.241250 21047 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:33.241344 21047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:20:33.241608 21047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:20:33.244424 21047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "069f3433cdbc43809f1b2dae4c508633"
format_stamp: "Formatted at 2026-08-09 20:20:33 on dist-test-slave-6k22"
server_key: "ca90290dcc7c07225eed89c063e3596d"
server_key_iv: "94080bb351712dc5e4a5a060070d103f"
server_key_version: "encryptionkey@0"
I20260809 20:20:33.245118 21047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "069f3433cdbc43809f1b2dae4c508633"
format_stamp: "Formatted at 2026-08-09 20:20:33 on dist-test-slave-6k22"
server_key: "ca90290dcc7c07225eed89c063e3596d"
server_key_iv: "94080bb351712dc5e4a5a060070d103f"
server_key_version: "encryptionkey@0"
I20260809 20:20:33.250126 21047 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260809 20:20:33.253436 21065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:33.254792 21047 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 20:20:33.255005 21047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "069f3433cdbc43809f1b2dae4c508633"
format_stamp: "Formatted at 2026-08-09 20:20:33 on dist-test-slave-6k22"
server_key: "ca90290dcc7c07225eed89c063e3596d"
server_key_iv: "94080bb351712dc5e4a5a060070d103f"
server_key_version: "encryptionkey@0"
I20260809 20:20:33.255257 21047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:33.265748 21047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:33.269618 21047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:33.269984 21047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:33.288399 21047 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:41961
I20260809 20:20:33.288439 21125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:41961 every 8 connection(s)
I20260809 20:20:33.289703 21047 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:20:33.293449 21126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:20:33.300029 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 21047
I20260809 20:20:33.300276 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:20:33.300987 5364 external_mini_cluster.cc:1428] Setting key e0ba0327e6562d0874c7a3ea49c97347
I20260809 20:20:33.304200 21126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 069f3433cdbc43809f1b2dae4c508633: Bootstrap starting.
I20260809 20:20:33.308252 21126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 069f3433cdbc43809f1b2dae4c508633: Neither blocks nor log segments found. Creating new log.
I20260809 20:20:33.309444 21126 log.cc:824] T 00000000000000000000000000000000 P 069f3433cdbc43809f1b2dae4c508633: Log is configured to *not* fsync() on all Append() calls
I20260809 20:20:33.312412 21126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 069f3433cdbc43809f1b2dae4c508633: No bootstrap required, opened a new log
Aug 09 20:20:33 dist-test-slave-6k22 krb5kdc[21032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:20:33.321789 21126 raft_consensus.cc:348] T 00000000000000000000000000000000 P 069f3433cdbc43809f1b2dae4c508633 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069f3433cdbc43809f1b2dae4c508633" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41961 } }
I20260809 20:20:33.322178 21126 raft_consensus.cc:374] T 00000000000000000000000000000000 P 069f3433cdbc43809f1b2dae4c508633 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:20:33.322319 21126 raft_consensus.cc:729] T 00000000000000000000000000000000 P 069f3433cdbc43809f1b2dae4c508633 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 069f3433cdbc43809f1b2dae4c508633, State: Initialized, Role: FOLLOWER
I20260809 20:20:33.322992 21126 consensus_queue.cc:260] T 00000000000000000000000000000000 P 069f3433cdbc43809f1b2dae4c508633 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069f3433cdbc43809f1b2dae4c508633" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41961 } }
I20260809 20:20:33.323258 21126 raft_consensus.cc:388] T 00000000000000000000000000000000 P 069f3433cdbc43809f1b2dae4c508633 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:20:33.323429 21126 raft_consensus.cc:482] T 00000000000000000000000000000000 P 069f3433cdbc43809f1b2dae4c508633 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:20:33.323669 21126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 069f3433cdbc43809f1b2dae4c508633 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:20:33.325861 21129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:33.302977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50364 (local address 127.5.61.62:41961)
0809 20:20:33.303891 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:33.303909 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:33.303961 (+ 52us) server_negotiation.cc:407] Connection header received
0809 20:20:33.306484 (+ 2523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:33.306511 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:33.307020 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:33.307464 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:33.308859 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:33.310663 (+ 1804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:33.312127 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:33.313186 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:33.313396 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:33.317456 (+ 4060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:33.317466 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:33.317489 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:33.317531 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:33.321520 (+ 3989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:33.322212 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:33.322219 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:33.322224 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:33.322475 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:33.322918 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:33.322923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:33.322926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:33.323629 (+ 703us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:33.323977 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:33.324445 (+ 468us) server_negotiation.cc:299] Negotiation successful
0809 20:20:33.324886 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":390,"threads_started":1}
I20260809 20:20:33.326159 21126 raft_consensus.cc:504] T 00000000000000000000000000000000 P 069f3433cdbc43809f1b2dae4c508633 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069f3433cdbc43809f1b2dae4c508633" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41961 } }
I20260809 20:20:33.326664 21126 leader_election.cc:302] T 00000000000000000000000000000000 P 069f3433cdbc43809f1b2dae4c508633 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 069f3433cdbc43809f1b2dae4c508633; no voters:
I20260809 20:20:33.327335 21126 leader_election.cc:288] T 00000000000000000000000000000000 P 069f3433cdbc43809f1b2dae4c508633 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:20:33.327527 21131 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 069f3433cdbc43809f1b2dae4c508633 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:20:33.329035 21131 raft_consensus.cc:686] T 00000000000000000000000000000000 P 069f3433cdbc43809f1b2dae4c508633 [term 1 LEADER]: Becoming Leader. State: Replica: 069f3433cdbc43809f1b2dae4c508633, State: Running, Role: LEADER
I20260809 20:20:33.329766 21126 sys_catalog.cc:564] T 00000000000000000000000000000000 P 069f3433cdbc43809f1b2dae4c508633 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:20:33.329649 21131 consensus_queue.cc:237] T 00000000000000000000000000000000 P 069f3433cdbc43809f1b2dae4c508633 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069f3433cdbc43809f1b2dae4c508633" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41961 } }
I20260809 20:20:33.330160 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:20:33.337037 21133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 069f3433cdbc43809f1b2dae4c508633 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 069f3433cdbc43809f1b2dae4c508633. Latest consensus state: current_term: 1 leader_uuid: "069f3433cdbc43809f1b2dae4c508633" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069f3433cdbc43809f1b2dae4c508633" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41961 } } }
I20260809 20:20:33.337152 21132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 069f3433cdbc43809f1b2dae4c508633 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "069f3433cdbc43809f1b2dae4c508633" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069f3433cdbc43809f1b2dae4c508633" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41961 } } }
I20260809 20:20:33.337558 21133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 069f3433cdbc43809f1b2dae4c508633 [sys.catalog]: This master's current role is: LEADER
I20260809 20:20:33.337567 21132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 069f3433cdbc43809f1b2dae4c508633 [sys.catalog]: This master's current role is: LEADER
I20260809 20:20:33.340574 21141 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:20:33.345418 21141 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:20:33.354009 21141 catalog_manager.cc:1347] Generated new cluster ID: 66f779736b1943428bbbce2341be878e
I20260809 20:20:33.354177 21141 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:20:33.377751 21141 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:20:33.379104 21141 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:20:33.385738 21141 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 069f3433cdbc43809f1b2dae4c508633: Generated new TSK 0
I20260809 20:20:33.386555 21141 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:20:33.442571 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 21047
I20260809 20:20:33.443842 21121 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:20:33.540880 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 21047
2026-08-09T20:20:33Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2075 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:20:33 dist-test-slave-6k22 krb5kdc[21155](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:20:33 dist-test-slave-6k22 krb5kdc[21155](info): set up 2 sockets
Aug 09 20:20:33 dist-test-slave-6k22 krb5kdc[21155](info): commencing operation
krb5kdc: starting...
W20260809 20:20:34.719015 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:20:34 dist-test-slave-6k22 krb5kdc[21155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:20:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:20:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:20:34.971274 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:46463
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:32969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:46463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:20:35.304919 21169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:35.305230 21169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:35.305392 21169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:20:35.312353 21169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:20:35.312481 21169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:35.312580 21169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:35.312701 21169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:20:35.312777 21169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:20:35.321918 21169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:32969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:46463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:46463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:35.323150 21169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:35.324482 21169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:35.336107 21178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:35.336169 21177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:35.336144 21180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:35.337103 21169 server_base.cc:1034] running on GCE node
I20260809 20:20:35.337847 21169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:35.339169 21169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:35.340404 21169 hybrid_clock.cc:648] HybridClock initialized: now 1786306835340369 us; error 44 us; skew 500 ppm
I20260809 20:20:35.340761 21169 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:20:35 dist-test-slave-6k22 krb5kdc[21155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306835, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:35.343889 21169 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:20:35.345316 21169 webserver.cc:460] Webserver started at http://127.5.61.62:35987/ using document root <none> and password file <none>
I20260809 20:20:35.345993 21169 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:35.346143 21169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:20:35.346451 21169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:20:35.349375 21169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "8c69b8e666d245ada106cc1db1ac8477"
format_stamp: "Formatted at 2026-08-09 20:20:35 on dist-test-slave-6k22"
server_key: "2e07a2de0c7414fd9eb5f6683382fc64"
server_key_iv: "c1133fc7cd642091af6bf4a9ec263ce4"
server_key_version: "encryptionkey@0"
I20260809 20:20:35.350173 21169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "8c69b8e666d245ada106cc1db1ac8477"
format_stamp: "Formatted at 2026-08-09 20:20:35 on dist-test-slave-6k22"
server_key: "2e07a2de0c7414fd9eb5f6683382fc64"
server_key_iv: "c1133fc7cd642091af6bf4a9ec263ce4"
server_key_version: "encryptionkey@0"
I20260809 20:20:35.355314 21169 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 20:20:35.358651 21189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:35.359972 21169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:20:35.360152 21169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "8c69b8e666d245ada106cc1db1ac8477"
format_stamp: "Formatted at 2026-08-09 20:20:35 on dist-test-slave-6k22"
server_key: "2e07a2de0c7414fd9eb5f6683382fc64"
server_key_iv: "c1133fc7cd642091af6bf4a9ec263ce4"
server_key_version: "encryptionkey@0"
I20260809 20:20:35.360342 21169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:35.377671 21169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:35.381595 21169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:35.381870 21169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:35.400485 21169 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:46463
I20260809 20:20:35.400506 21244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:46463 every 8 connection(s)
I20260809 20:20:35.401854 21169 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:20:35.405599 21245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:20:35.407069 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 21169
I20260809 20:20:35.407311 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:20:35.408170 5364 external_mini_cluster.cc:1428] Setting key 042d88f4265e3ed7b49fdc4219a8d64e
I20260809 20:20:35.420275 21245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477: Bootstrap starting.
Aug 09 20:20:35 dist-test-slave-6k22 krb5kdc[21155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:20:35.424166 21245 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477: Neither blocks nor log segments found. Creating new log.
I20260809 20:20:35.425324 21245 log.cc:824] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477: Log is configured to *not* fsync() on all Append() calls
I20260809 20:20:35.428289 21245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477: No bootstrap required, opened a new log
I20260809 20:20:35.431900 21249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:35.410227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44496 (local address 127.5.61.62:46463)
0809 20:20:35.411029 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:35.411040 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:35.411082 (+ 42us) server_negotiation.cc:407] Connection header received
0809 20:20:35.412658 (+ 1576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:35.412678 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:35.413050 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:35.413372 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:35.415135 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:35.416566 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:35.418571 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:35.419223 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:35.419359 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:35.424373 (+ 5014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:35.424381 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:35.424394 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:35.424429 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:35.427607 (+ 3178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:35.428346 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:35.428354 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:35.428364 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:35.428568 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:35.428966 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:35.428971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:35.428974 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:35.429533 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:35.429853 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:35.430317 (+ 464us) server_negotiation.cc:299] Negotiation successful
0809 20:20:35.430765 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":579,"thread_start_us":376,"threads_started":1}
I20260809 20:20:35.436298 21245 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c69b8e666d245ada106cc1db1ac8477" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46463 } }
I20260809 20:20:35.436658 21245 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:20:35.436769 21245 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c69b8e666d245ada106cc1db1ac8477, State: Initialized, Role: FOLLOWER
I20260809 20:20:35.437404 21245 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c69b8e666d245ada106cc1db1ac8477" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46463 } }
I20260809 20:20:35.437657 21245 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:20:35.437842 21245 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:20:35.438019 21245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:20:35.440243 21245 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c69b8e666d245ada106cc1db1ac8477" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46463 } }
I20260809 20:20:35.440722 21245 leader_election.cc:302] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c69b8e666d245ada106cc1db1ac8477; no voters:
I20260809 20:20:35.441304 21245 leader_election.cc:288] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:20:35.441433 21251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:20:35.442173 21251 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 1 LEADER]: Becoming Leader. State: Replica: 8c69b8e666d245ada106cc1db1ac8477, State: Running, Role: LEADER
I20260809 20:20:35.442639 21251 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c69b8e666d245ada106cc1db1ac8477" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46463 } }
I20260809 20:20:35.443305 21245 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:20:35.445760 21253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c69b8e666d245ada106cc1db1ac8477. Latest consensus state: current_term: 1 leader_uuid: "8c69b8e666d245ada106cc1db1ac8477" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c69b8e666d245ada106cc1db1ac8477" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46463 } } }
I20260809 20:20:35.445843 21252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c69b8e666d245ada106cc1db1ac8477" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c69b8e666d245ada106cc1db1ac8477" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46463 } } }
I20260809 20:20:35.446312 21253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [sys.catalog]: This master's current role is: LEADER
I20260809 20:20:35.446372 21252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [sys.catalog]: This master's current role is: LEADER
I20260809 20:20:35.448659 21259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:20:35.454762 21259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:20:35.465678 21259 catalog_manager.cc:1347] Generated new cluster ID: 84b3bfbcdc594f2db1e6498ba6dfebc9
I20260809 20:20:35.465798 21259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:20:35.485862 21259 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:20:35.487459 21259 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:20:35.500759 21259 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477: Generated new TSK 0
I20260809 20:20:35.501816 21259 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.61.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
I20260809 20:20:35.568154 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:46463
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:32969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:20:35.901479 21273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:35.901779 21273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:35.901942 21273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:20:35.909552 21273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:35.909700 21273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:35.909889 21273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:20:35.919764 21273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:32969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:46463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:35.921025 21273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:35.922420 21273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:35.935930 21279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:35.936400 21282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:35.936627 21278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:35.937897 21273 server_base.cc:1034] running on GCE node
I20260809 20:20:35.938640 21273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:35.939627 21273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:35.940980 21273 hybrid_clock.cc:648] HybridClock initialized: now 1786306835940918 us; error 46 us; skew 500 ppm
I20260809 20:20:35.941327 21273 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:20:35 dist-test-slave-6k22 krb5kdc[21155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306835, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:35.944546 21273 init.cc:377] Logged in from keytab as kudu/127.5.61.1@KRBTEST.COM (short username kudu)
I20260809 20:20:35.945955 21273 webserver.cc:460] Webserver started at http://127.5.61.1:41785/ using document root <none> and password file <none>
I20260809 20:20:35.946641 21273 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:35.946797 21273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:20:35.947093 21273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:20:35.949949 21273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "16db83d271ac4c2fa0f13fae0e6339d7"
format_stamp: "Formatted at 2026-08-09 20:20:35 on dist-test-slave-6k22"
server_key: "410044219cbfc5ecfd1e5611a15035d6"
server_key_iv: "fb6ffcf585f70f4358959337f2bcbeaa"
server_key_version: "encryptionkey@0"
I20260809 20:20:35.950673 21273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "16db83d271ac4c2fa0f13fae0e6339d7"
format_stamp: "Formatted at 2026-08-09 20:20:35 on dist-test-slave-6k22"
server_key: "410044219cbfc5ecfd1e5611a15035d6"
server_key_iv: "fb6ffcf585f70f4358959337f2bcbeaa"
server_key_version: "encryptionkey@0"
I20260809 20:20:35.955802 21273 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 20:20:35.959124 21290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:35.960424 21273 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:20:35.960613 21273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "16db83d271ac4c2fa0f13fae0e6339d7"
format_stamp: "Formatted at 2026-08-09 20:20:35 on dist-test-slave-6k22"
server_key: "410044219cbfc5ecfd1e5611a15035d6"
server_key_iv: "fb6ffcf585f70f4358959337f2bcbeaa"
server_key_version: "encryptionkey@0"
I20260809 20:20:35.960825 21273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:35.986685 21273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:35.990571 21273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:35.990893 21273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:35.991870 21273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:20:35.993137 21273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:20:35.993247 21273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:35.993388 21273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:20:35.993458 21273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:36.026757 21273 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:45771
I20260809 20:20:36.026847 21413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:45771 every 8 connection(s)
I20260809 20:20:36.028076 21273 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:20:36.035709 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 21273
I20260809 20:20:36.035938 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:20:36.036607 5364 external_mini_cluster.cc:1428] Setting key 6b2a6e0bb695efc6d7347c3b8b7a1ffc
Aug 09 20:20:36 dist-test-slave-6k22 krb5kdc[21155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306835, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:20:36.048619 21417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:36.030709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:42691 (local address 127.5.61.62:46463)
0809 20:20:36.031253 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:36.031257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:36.032180 (+ 923us) server_negotiation.cc:407] Connection header received
0809 20:20:36.033970 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:36.033975 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:36.034064 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:36.034196 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:36.035632 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:36.036470 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:36.038272 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:36.038693 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:36.038837 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:36.042548 (+ 3711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:36.042553 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:36.042556 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:36.042587 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:36.045399 (+ 2812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:36.046329 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:36.046332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:36.046335 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:36.046455 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:36.046927 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:36.046930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:36.046932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:36.047181 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:36.047306 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:36.048210 (+ 904us) server_negotiation.cc:299] Negotiation successful
0809 20:20:36.048373 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":329,"threads_started":1}
I20260809 20:20:36.049939 21418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:36.031204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.1:42691)
0809 20:20:36.031998 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:36.032043 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:36.033713 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:36.034417 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:36.034425 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:36.034925 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:20:36.035451 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:36.035467 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:36.036651 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:36.036656 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:36.038120 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:36.038132 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:36.038919 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:36.038927 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:36.039158 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:36.039881 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:20:36.039902 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:20:36.042337 (+ 2435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:20:36.045537 (+ 3200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:36.045548 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:36.045565 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:36.046199 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:36.046562 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:36.046569 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:36.046576 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:36.046818 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:36.047311 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:20:36.047322 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:20:36.047789 (+ 467us) client_negotiation.cc:769] Sending connection context
0809 20:20:36.048108 (+ 319us) client_negotiation.cc:240] Negotiation successful
0809 20:20:36.048664 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":339,"threads_started":1}
I20260809 20:20:36.051478 21414 heartbeater.cc:347] Connected to a master server at 127.5.61.62:46463
I20260809 20:20:36.051811 21414 heartbeater.cc:464] Registering TS with master...
I20260809 20:20:36.052624 21414 heartbeater.cc:511] Master 127.5.61.62:46463 requested a full tablet report, sending...
I20260809 20:20:36.054831 21208 ts_manager.cc:194] Registered new tserver with Master: 16db83d271ac4c2fa0f13fae0e6339d7 (127.5.61.1:45771)
I20260809 20:20:36.056280 21208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.1@KRBTEST.COM'} at 127.5.61.1:42691
WARNING: no policy specified for kudu/127.5.61.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
I20260809 20:20:36.098482 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:46463
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:32969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:20:36.434288 21421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:36.434615 21421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:36.434803 21421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:20:36.441815 21421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:36.441969 21421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:36.442139 21421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:20:36.451459 21421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:32969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:46463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:36.453006 21421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:36.454285 21421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:36.469002 21429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:36.469074 21427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:36.469091 21426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:36.469846 21421 server_base.cc:1034] running on GCE node
I20260809 20:20:36.470453 21421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:36.471359 21421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:36.472571 21421 hybrid_clock.cc:648] HybridClock initialized: now 1786306836472503 us; error 51 us; skew 500 ppm
I20260809 20:20:36.472992 21421 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:20:36 dist-test-slave-6k22 krb5kdc[21155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306836, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:36.476293 21421 init.cc:377] Logged in from keytab as kudu/127.5.61.2@KRBTEST.COM (short username kudu)
I20260809 20:20:36.477778 21421 webserver.cc:460] Webserver started at http://127.5.61.2:40065/ using document root <none> and password file <none>
I20260809 20:20:36.478478 21421 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:36.478614 21421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:20:36.478952 21421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:20:36.482029 21421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "3bd444f3769b4452a20f74b5341c1ac2"
format_stamp: "Formatted at 2026-08-09 20:20:36 on dist-test-slave-6k22"
server_key: "53ad4d1cac2ccc6db4b91968915fb390"
server_key_iv: "e252fc5413065112e812f8de2bc48976"
server_key_version: "encryptionkey@0"
I20260809 20:20:36.482787 21421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "3bd444f3769b4452a20f74b5341c1ac2"
format_stamp: "Formatted at 2026-08-09 20:20:36 on dist-test-slave-6k22"
server_key: "53ad4d1cac2ccc6db4b91968915fb390"
server_key_iv: "e252fc5413065112e812f8de2bc48976"
server_key_version: "encryptionkey@0"
I20260809 20:20:36.488068 21421 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 20:20:36.491444 21438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:36.492744 21421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:20:36.492905 21421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "3bd444f3769b4452a20f74b5341c1ac2"
format_stamp: "Formatted at 2026-08-09 20:20:36 on dist-test-slave-6k22"
server_key: "53ad4d1cac2ccc6db4b91968915fb390"
server_key_iv: "e252fc5413065112e812f8de2bc48976"
server_key_version: "encryptionkey@0"
I20260809 20:20:36.493108 21421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:36.518627 21421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:36.523124 21421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:36.523440 21421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:36.524559 21421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:20:36.525930 21421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:20:36.526005 21421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:36.526175 21421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:20:36.526227 21421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:36.560762 21421 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:34751
I20260809 20:20:36.560797 21558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:34751 every 8 connection(s)
I20260809 20:20:36.562049 21421 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:20:36.564702 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 21421
I20260809 20:20:36.564939 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:20:36.565538 5364 external_mini_cluster.cc:1428] Setting key 798767368606e6479e933342bb7599ba
Aug 09 20:20:36 dist-test-slave-6k22 krb5kdc[21155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306836, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:20:36.585031 21562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:36.564801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:58129 (local address 127.5.61.62:46463)
0809 20:20:36.565364 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:36.565371 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:36.566476 (+ 1105us) server_negotiation.cc:407] Connection header received
0809 20:20:36.568924 (+ 2448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:36.568928 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:36.568990 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:36.569083 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:36.571195 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:36.571996 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:36.574408 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:36.574813 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:36.574905 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:36.579055 (+ 4150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:36.579060 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:36.579063 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:36.579094 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:36.582066 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:36.582765 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:36.582770 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:36.582773 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:36.582902 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:36.583356 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:36.583364 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:36.583368 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:36.583689 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:36.583867 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:36.584472 (+ 605us) server_negotiation.cc:299] Negotiation successful
0809 20:20:36.584680 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":298,"threads_started":1}
I20260809 20:20:36.585940 21563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:36.565386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.2:58129)
0809 20:20:36.566305 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:36.566355 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:36.568646 (+ 2291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:36.569338 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:36.569351 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:36.570167 (+ 816us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:20:36.571002 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:36.571030 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:36.572143 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:36.572151 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:36.574187 (+ 2036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:36.574206 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:36.575034 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:36.575042 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:36.575342 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:36.576105 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:20:36.576137 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:20:36.578832 (+ 2695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:20:36.582220 (+ 3388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:36.582227 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:36.582242 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:36.582636 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:36.583014 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:36.583019 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:36.583022 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:36.583234 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:36.583816 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:20:36.583824 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:20:36.584135 (+ 311us) client_negotiation.cc:769] Sending connection context
0809 20:20:36.584383 (+ 248us) client_negotiation.cc:240] Negotiation successful
0809 20:20:36.584871 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":379,"threads_started":1}
I20260809 20:20:36.587625 21559 heartbeater.cc:347] Connected to a master server at 127.5.61.62:46463
I20260809 20:20:36.587926 21559 heartbeater.cc:464] Registering TS with master...
I20260809 20:20:36.588603 21559 heartbeater.cc:511] Master 127.5.61.62:46463 requested a full tablet report, sending...
I20260809 20:20:36.590121 21208 ts_manager.cc:194] Registered new tserver with Master: 3bd444f3769b4452a20f74b5341c1ac2 (127.5.61.2:34751)
I20260809 20:20:36.591384 21208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.2@KRBTEST.COM'} at 127.5.61.2:58129
WARNING: no policy specified for kudu/127.5.61.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
I20260809 20:20:36.624701 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:46463
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:32969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:20:36.962647 21566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:36.962937 21566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:36.963042 21566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:20:36.970635 21566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:36.970748 21566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:36.970865 21566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:20:36.980618 21566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:32969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:46463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:36.981954 21566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:36.983294 21566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:36.996876 21572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:36.996840 21574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:36.997016 21571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:36.997614 21566 server_base.cc:1034] running on GCE node
I20260809 20:20:36.998265 21566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:36.999193 21566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:37.000416 21566 hybrid_clock.cc:648] HybridClock initialized: now 1786306837000363 us; error 62 us; skew 500 ppm
I20260809 20:20:37.000725 21566 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:20:36 dist-test-slave-6k22 krb5kdc[21155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306836, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:37.003898 21566 init.cc:377] Logged in from keytab as kudu/127.5.61.3@KRBTEST.COM (short username kudu)
I20260809 20:20:37.005321 21566 webserver.cc:460] Webserver started at http://127.5.61.3:38121/ using document root <none> and password file <none>
I20260809 20:20:37.005959 21566 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:37.006057 21566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:20:37.006318 21566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:20:37.009229 21566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "8e6be863daab4c0da5d4e268a24f4ca0"
format_stamp: "Formatted at 2026-08-09 20:20:37 on dist-test-slave-6k22"
server_key: "4f5abad1086689b31f9c9c9beac7a159"
server_key_iv: "a9e34eea7de7dd903398d049a504b575"
server_key_version: "encryptionkey@0"
I20260809 20:20:37.010087 21566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "8e6be863daab4c0da5d4e268a24f4ca0"
format_stamp: "Formatted at 2026-08-09 20:20:37 on dist-test-slave-6k22"
server_key: "4f5abad1086689b31f9c9c9beac7a159"
server_key_iv: "a9e34eea7de7dd903398d049a504b575"
server_key_version: "encryptionkey@0"
I20260809 20:20:37.015130 21566 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 20:20:37.018540 21581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:37.019851 21566 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:20:37.020015 21566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "8e6be863daab4c0da5d4e268a24f4ca0"
format_stamp: "Formatted at 2026-08-09 20:20:37 on dist-test-slave-6k22"
server_key: "4f5abad1086689b31f9c9c9beac7a159"
server_key_iv: "a9e34eea7de7dd903398d049a504b575"
server_key_version: "encryptionkey@0"
I20260809 20:20:37.020203 21566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:37.032289 21566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:37.036397 21566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:37.036688 21566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:37.037595 21566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:20:37.038882 21566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:20:37.038983 21566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:37.039132 21566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:20:37.039208 21566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:37.059829 21414 heartbeater.cc:503] Master 127.5.61.62:46463 was elected leader, sending a full tablet report...
I20260809 20:20:37.073436 21566 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:46119
I20260809 20:20:37.073486 21701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:46119 every 8 connection(s)
I20260809 20:20:37.074664 21566 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:20:37.077713 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 21566
I20260809 20:20:37.077914 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:20:37.078600 5364 external_mini_cluster.cc:1428] Setting key 657090fb224ca39935b6b6b1c0ed8b73
Aug 09 20:20:37 dist-test-slave-6k22 krb5kdc[21155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306836, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:20:37.094359 21562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:37.077278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:52943 (local address 127.5.61.62:46463)
0809 20:20:37.077464 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:37.077469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:37.079070 (+ 1601us) server_negotiation.cc:407] Connection header received
0809 20:20:37.081222 (+ 2152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:37.081228 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:37.081322 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:37.081432 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:37.082903 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.083732 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.085122 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.085559 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.085693 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:37.089129 (+ 3436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:37.089133 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:37.089136 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:37.089171 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:37.091500 (+ 2329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:37.092223 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:37.092227 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:37.092229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:37.092343 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:37.092750 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:37.092756 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:37.092759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:37.093020 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:37.093200 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:37.093812 (+ 612us) server_negotiation.cc:299] Negotiation successful
0809 20:20:37.094040 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 20:20:37.095206 21704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:37.077941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.3:52943)
0809 20:20:37.078915 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:37.078959 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:37.080958 (+ 1999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:37.081656 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:37.081671 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:37.082143 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:20:37.082741 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:37.082757 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.083875 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.083879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:37.084987 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:37.084997 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.085795 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.085801 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:37.086006 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:37.086736 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:20:37.086757 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:20:37.088937 (+ 2180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:20:37.091692 (+ 2755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:37.091699 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:37.091713 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:37.092076 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:37.092449 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:37.092453 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:37.092456 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:37.092639 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:37.093141 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:20:37.093150 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:20:37.093447 (+ 297us) client_negotiation.cc:769] Sending connection context
0809 20:20:37.093719 (+ 272us) client_negotiation.cc:240] Negotiation successful
0809 20:20:37.094192 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":326,"threads_started":1}
I20260809 20:20:37.096449 21702 heartbeater.cc:347] Connected to a master server at 127.5.61.62:46463
I20260809 20:20:37.096735 21702 heartbeater.cc:464] Registering TS with master...
I20260809 20:20:37.097415 21702 heartbeater.cc:511] Master 127.5.61.62:46463 requested a full tablet report, sending...
I20260809 20:20:37.098802 21208 ts_manager.cc:194] Registered new tserver with Master: 8e6be863daab4c0da5d4e268a24f4ca0 (127.5.61.3:46119)
I20260809 20:20:37.099545 21208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.3@KRBTEST.COM'} at 127.5.61.3:52943
I20260809 20:20:37.105691 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 20:20:37.122854 21562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:37.110383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44512 (local address 127.5.61.62:46463)
0809 20:20:37.110606 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:37.110615 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:37.110946 (+ 331us) server_negotiation.cc:407] Connection header received
0809 20:20:37.111150 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:37.111155 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:37.111226 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:37.111329 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:37.112748 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.113466 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.114757 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.115196 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.115329 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:37.117746 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:37.117753 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:37.117758 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:37.117797 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:37.120188 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:37.120834 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:37.120840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:37.120844 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:37.120972 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:37.121349 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:37.121354 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:37.121358 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:37.121674 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:37.121948 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:37.122243 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 20:20:37.122475 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 20:20:37.126655 21208 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:20:37.133873 21208 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:20:37.156559 21714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:37.146475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:46119 (local address 127.0.0.1:55806)
0809 20:20:37.147219 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:37.147275 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:37.147452 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:37.148755 (+ 1303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:37.148762 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:37.148796 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:20:37.149249 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:37.149261 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.151145 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.151150 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:37.153710 (+ 2560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:37.153726 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.155625 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.155632 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:37.155825 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:37.155879 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 20:20:37.155976 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 20:20:37.156165 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":761,"threads_started":1}
I20260809 20:20:37.157115 21715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:37.146554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:34751 (local address 127.0.0.1:47872)
0809 20:20:37.148808 (+ 2254us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:37.148823 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:37.148955 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:37.149796 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:37.149800 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:37.149824 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:20:37.150237 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:37.150249 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.152026 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.152031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:37.154215 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:37.154228 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.156334 (+ 2106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.156340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:37.156505 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:37.156554 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 20:20:37.156642 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 20:20:37.156772 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2123,"thread_start_us":1997,"threads_started":1}
I20260809 20:20:37.157821 21716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:37.146814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47872 (local address 127.5.61.2:34751)
0809 20:20:37.148436 (+ 1622us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:37.148443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:37.149072 (+ 629us) server_negotiation.cc:407] Connection header received
0809 20:20:37.149174 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:37.149181 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:37.149396 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:37.149574 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:20:37.150437 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.151855 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.154421 (+ 2566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.155873 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.155995 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:37.156181 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:37.157325 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0809 20:20:37.157504 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1454,"thread_start_us":1361,"threads_started":1}
I20260809 20:20:37.158699 21717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:37.146942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55806 (local address 127.5.61.3:46119)
0809 20:20:37.148179 (+ 1237us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:37.148189 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:37.148224 (+ 35us) server_negotiation.cc:407] Connection header received
0809 20:20:37.148337 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:37.148344 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:37.148560 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:37.148755 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:20:37.149471 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.151006 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.153918 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.155352 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.157712 (+ 2360us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:37.158034 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:37.158129 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 20:20:37.158280 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1044,"thread_start_us":881,"threads_started":1}
I20260809 20:20:37.162806 21633 tablet_service.cc:1467] Processing CreateTablet for tablet 87769642c2964fba856bf07a2dfab9df (DEFAULT_TABLE table=test-table [id=800e458a01d04a5f8fbbc23fb3bf6310]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:20:37.162806 21491 tablet_service.cc:1467] Processing CreateTablet for tablet 87769642c2964fba856bf07a2dfab9df (DEFAULT_TABLE table=test-table [id=800e458a01d04a5f8fbbc23fb3bf6310]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:20:37.164309 21633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87769642c2964fba856bf07a2dfab9df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:20:37.164304 21491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87769642c2964fba856bf07a2dfab9df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:20:37.165400 21714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:37.148906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:45771 (local address 127.0.0.1:57062)
0809 20:20:37.157446 (+ 8540us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:37.157463 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:37.157556 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:37.159432 (+ 1876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:37.159437 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:37.159463 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:20:37.159650 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:37.159664 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.161439 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.161446 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:37.162817 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:37.162831 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.164646 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.164651 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:37.164817 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:37.164920 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 20:20:37.165013 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 20:20:37.165148 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8413,"thread_start_us":9961,"threads_started":1}
I20260809 20:20:37.165539 21719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:37.149482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57062 (local address 127.5.61.1:45771)
0809 20:20:37.158839 (+ 9357us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:37.158847 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:37.158886 (+ 39us) server_negotiation.cc:407] Connection header received
0809 20:20:37.158991 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:37.158998 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:37.159216 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:37.159876 (+ 660us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:20:37.160151 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.161289 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.163001 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.164479 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.164619 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:37.164884 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:37.165064 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 20:20:37.165206 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9163,"thread_start_us":8985,"threads_started":1}
I20260809 20:20:37.168686 21343 tablet_service.cc:1467] Processing CreateTablet for tablet 87769642c2964fba856bf07a2dfab9df (DEFAULT_TABLE table=test-table [id=800e458a01d04a5f8fbbc23fb3bf6310]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:20:37.169768 21343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87769642c2964fba856bf07a2dfab9df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:20:37.179464 21720 tablet_bootstrap.cc:492] T 87769642c2964fba856bf07a2dfab9df P 8e6be863daab4c0da5d4e268a24f4ca0: Bootstrap starting.
I20260809 20:20:37.181789 21722 tablet_bootstrap.cc:492] T 87769642c2964fba856bf07a2dfab9df P 16db83d271ac4c2fa0f13fae0e6339d7: Bootstrap starting.
I20260809 20:20:37.184329 21721 tablet_bootstrap.cc:492] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2: Bootstrap starting.
I20260809 20:20:37.184778 21720 tablet_bootstrap.cc:654] T 87769642c2964fba856bf07a2dfab9df P 8e6be863daab4c0da5d4e268a24f4ca0: Neither blocks nor log segments found. Creating new log.
I20260809 20:20:37.185623 21722 tablet_bootstrap.cc:654] T 87769642c2964fba856bf07a2dfab9df P 16db83d271ac4c2fa0f13fae0e6339d7: Neither blocks nor log segments found. Creating new log.
I20260809 20:20:37.186234 21720 log.cc:824] T 87769642c2964fba856bf07a2dfab9df P 8e6be863daab4c0da5d4e268a24f4ca0: Log is configured to *not* fsync() on all Append() calls
I20260809 20:20:37.186661 21722 log.cc:824] T 87769642c2964fba856bf07a2dfab9df P 16db83d271ac4c2fa0f13fae0e6339d7: Log is configured to *not* fsync() on all Append() calls
I20260809 20:20:37.187886 21721 tablet_bootstrap.cc:654] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2: Neither blocks nor log segments found. Creating new log.
I20260809 20:20:37.189133 21721 log.cc:824] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2: Log is configured to *not* fsync() on all Append() calls
I20260809 20:20:37.189429 21720 tablet_bootstrap.cc:492] T 87769642c2964fba856bf07a2dfab9df P 8e6be863daab4c0da5d4e268a24f4ca0: No bootstrap required, opened a new log
I20260809 20:20:37.189662 21720 ts_tablet_manager.cc:1397] T 87769642c2964fba856bf07a2dfab9df P 8e6be863daab4c0da5d4e268a24f4ca0: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260809 20:20:37.190178 21722 tablet_bootstrap.cc:492] T 87769642c2964fba856bf07a2dfab9df P 16db83d271ac4c2fa0f13fae0e6339d7: No bootstrap required, opened a new log
I20260809 20:20:37.190414 21722 ts_tablet_manager.cc:1397] T 87769642c2964fba856bf07a2dfab9df P 16db83d271ac4c2fa0f13fae0e6339d7: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260809 20:20:37.192101 21721 tablet_bootstrap.cc:492] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2: No bootstrap required, opened a new log
I20260809 20:20:37.192361 21721 ts_tablet_manager.cc:1397] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 20:20:37.198278 21722 raft_consensus.cc:348] T 87769642c2964fba856bf07a2dfab9df P 16db83d271ac4c2fa0f13fae0e6339d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6be863daab4c0da5d4e268a24f4ca0" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46119 } } peers { permanent_uuid: "3bd444f3769b4452a20f74b5341c1ac2" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34751 } } peers { permanent_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 45771 } }
I20260809 20:20:37.198660 21722 raft_consensus.cc:374] T 87769642c2964fba856bf07a2dfab9df P 16db83d271ac4c2fa0f13fae0e6339d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:20:37.198765 21722 raft_consensus.cc:729] T 87769642c2964fba856bf07a2dfab9df P 16db83d271ac4c2fa0f13fae0e6339d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16db83d271ac4c2fa0f13fae0e6339d7, State: Initialized, Role: FOLLOWER
I20260809 20:20:37.199343 21722 consensus_queue.cc:260] T 87769642c2964fba856bf07a2dfab9df P 16db83d271ac4c2fa0f13fae0e6339d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6be863daab4c0da5d4e268a24f4ca0" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46119 } } peers { permanent_uuid: "3bd444f3769b4452a20f74b5341c1ac2" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34751 } } peers { permanent_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 45771 } }
I20260809 20:20:37.200177 21720 raft_consensus.cc:348] T 87769642c2964fba856bf07a2dfab9df P 8e6be863daab4c0da5d4e268a24f4ca0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6be863daab4c0da5d4e268a24f4ca0" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46119 } } peers { permanent_uuid: "3bd444f3769b4452a20f74b5341c1ac2" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34751 } } peers { permanent_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 45771 } }
I20260809 20:20:37.200605 21720 raft_consensus.cc:374] T 87769642c2964fba856bf07a2dfab9df P 8e6be863daab4c0da5d4e268a24f4ca0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:20:37.200688 21720 raft_consensus.cc:729] T 87769642c2964fba856bf07a2dfab9df P 8e6be863daab4c0da5d4e268a24f4ca0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e6be863daab4c0da5d4e268a24f4ca0, State: Initialized, Role: FOLLOWER
I20260809 20:20:37.201150 21722 ts_tablet_manager.cc:1428] T 87769642c2964fba856bf07a2dfab9df P 16db83d271ac4c2fa0f13fae0e6339d7: Time spent starting tablet: real 0.011s user 0.008s sys 0.003s
I20260809 20:20:37.201339 21720 consensus_queue.cc:260] T 87769642c2964fba856bf07a2dfab9df P 8e6be863daab4c0da5d4e268a24f4ca0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6be863daab4c0da5d4e268a24f4ca0" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46119 } } peers { permanent_uuid: "3bd444f3769b4452a20f74b5341c1ac2" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34751 } } peers { permanent_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 45771 } }
I20260809 20:20:37.202359 21702 heartbeater.cc:503] Master 127.5.61.62:46463 was elected leader, sending a full tablet report...
I20260809 20:20:37.202706 21721 raft_consensus.cc:348] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6be863daab4c0da5d4e268a24f4ca0" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46119 } } peers { permanent_uuid: "3bd444f3769b4452a20f74b5341c1ac2" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34751 } } peers { permanent_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 45771 } }
I20260809 20:20:37.203125 21721 raft_consensus.cc:374] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:20:37.203226 21721 raft_consensus.cc:729] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd444f3769b4452a20f74b5341c1ac2, State: Initialized, Role: FOLLOWER
I20260809 20:20:37.204283 21720 ts_tablet_manager.cc:1428] T 87769642c2964fba856bf07a2dfab9df P 8e6be863daab4c0da5d4e268a24f4ca0: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260809 20:20:37.203841 21721 consensus_queue.cc:260] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6be863daab4c0da5d4e268a24f4ca0" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46119 } } peers { permanent_uuid: "3bd444f3769b4452a20f74b5341c1ac2" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34751 } } peers { permanent_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 45771 } }
I20260809 20:20:37.205931 21559 heartbeater.cc:503] Master 127.5.61.62:46463 was elected leader, sending a full tablet report...
I20260809 20:20:37.206552 21721 ts_tablet_manager.cc:1428] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260809 20:20:37.219146 21731 raft_consensus.cc:482] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:20:37.219367 21731 raft_consensus.cc:504] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6be863daab4c0da5d4e268a24f4ca0" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46119 } } peers { permanent_uuid: "3bd444f3769b4452a20f74b5341c1ac2" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34751 } } peers { permanent_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 45771 } }
I20260809 20:20:37.221181 21731 leader_election.cc:288] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e6be863daab4c0da5d4e268a24f4ca0 (127.5.61.3:46119), 16db83d271ac4c2fa0f13fae0e6339d7 (127.5.61.1:45771)
I20260809 20:20:37.229928 21717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:37.221616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:57751 (local address 127.5.61.3:46119)
0809 20:20:37.221811 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:37.221820 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:37.222330 (+ 510us) server_negotiation.cc:407] Connection header received
0809 20:20:37.222526 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:37.222531 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:37.222614 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:37.222707 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:20:37.223482 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.224587 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.227476 (+ 2889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.228725 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.228859 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:37.228961 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:37.229547 (+ 586us) server_negotiation.cc:299] Negotiation successful
0809 20:20:37.229674 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 20:20:37.229928 21719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:37.221664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:34205 (local address 127.5.61.1:45771)
0809 20:20:37.221919 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:37.221925 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:37.222469 (+ 544us) server_negotiation.cc:407] Connection header received
0809 20:20:37.222578 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:37.222583 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:37.222662 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:37.222739 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:20:37.223482 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.224587 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.227482 (+ 2895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.228754 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.228863 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:37.228961 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:37.229547 (+ 586us) server_negotiation.cc:299] Negotiation successful
0809 20:20:37.229674 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260809 20:20:37.230175 21732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:37.221412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:46119 (local address 127.5.61.2:57751)
0809 20:20:37.222201 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:37.222226 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:37.222382 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:37.222752 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:37.222756 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:37.222781 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:20:37.223316 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:37.223328 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.224713 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.224718 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:37.227319 (+ 2601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:37.227333 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.229019 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.229023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:37.229311 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:37.229409 (+ 98us) client_negotiation.cc:769] Sending connection context
0809 20:20:37.229504 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 20:20:37.229697 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":283,"thread_start_us":409,"threads_started":1}
I20260809 20:20:37.230422 21733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:37.221549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:45771 (local address 127.5.61.2:34205)
0809 20:20:37.222354 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:37.222376 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:37.222480 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:37.222765 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:37.222767 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:37.222844 (+ 77us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:20:37.223317 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:37.223328 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.224715 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.224718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:37.227318 (+ 2600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:37.227333 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.229013 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.229020 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:37.229314 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:37.229406 (+ 92us) client_negotiation.cc:769] Sending connection context
0809 20:20:37.229504 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 20:20:37.229697 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"mutex_wait_us":246,"spinlock_wait_cycles":16768,"thread_start_us":528,"threads_started":1}
I20260809 20:20:37.231168 21655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87769642c2964fba856bf07a2dfab9df" candidate_uuid: "3bd444f3769b4452a20f74b5341c1ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e6be863daab4c0da5d4e268a24f4ca0" is_pre_election: true
I20260809 20:20:37.231772 21655 raft_consensus.cc:2454] T 87769642c2964fba856bf07a2dfab9df P 8e6be863daab4c0da5d4e268a24f4ca0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bd444f3769b4452a20f74b5341c1ac2 in term 0.
I20260809 20:20:37.231710 21363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87769642c2964fba856bf07a2dfab9df" candidate_uuid: "3bd444f3769b4452a20f74b5341c1ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" is_pre_election: true
I20260809 20:20:37.232277 21363 raft_consensus.cc:2454] T 87769642c2964fba856bf07a2dfab9df P 16db83d271ac4c2fa0f13fae0e6339d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bd444f3769b4452a20f74b5341c1ac2 in term 0.
I20260809 20:20:37.232515 21439 leader_election.cc:302] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bd444f3769b4452a20f74b5341c1ac2, 8e6be863daab4c0da5d4e268a24f4ca0; no voters:
I20260809 20:20:37.232884 21731 raft_consensus.cc:2781] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:20:37.233038 21731 raft_consensus.cc:482] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:20:37.233157 21731 raft_consensus.cc:3037] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:20:37.235271 21731 raft_consensus.cc:504] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6be863daab4c0da5d4e268a24f4ca0" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46119 } } peers { permanent_uuid: "3bd444f3769b4452a20f74b5341c1ac2" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34751 } } peers { permanent_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 45771 } }
I20260809 20:20:37.235939 21731 leader_election.cc:288] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2 [CANDIDATE]: Term 1 election: Requested vote from peers 8e6be863daab4c0da5d4e268a24f4ca0 (127.5.61.3:46119), 16db83d271ac4c2fa0f13fae0e6339d7 (127.5.61.1:45771)
I20260809 20:20:37.236330 21655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87769642c2964fba856bf07a2dfab9df" candidate_uuid: "3bd444f3769b4452a20f74b5341c1ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e6be863daab4c0da5d4e268a24f4ca0"
I20260809 20:20:37.236529 21655 raft_consensus.cc:3037] T 87769642c2964fba856bf07a2dfab9df P 8e6be863daab4c0da5d4e268a24f4ca0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:20:37.236531 21363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87769642c2964fba856bf07a2dfab9df" candidate_uuid: "3bd444f3769b4452a20f74b5341c1ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16db83d271ac4c2fa0f13fae0e6339d7"
I20260809 20:20:37.236718 21363 raft_consensus.cc:3037] T 87769642c2964fba856bf07a2dfab9df P 16db83d271ac4c2fa0f13fae0e6339d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:20:37.238739 21655 raft_consensus.cc:2454] T 87769642c2964fba856bf07a2dfab9df P 8e6be863daab4c0da5d4e268a24f4ca0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bd444f3769b4452a20f74b5341c1ac2 in term 1.
I20260809 20:20:37.238909 21363 raft_consensus.cc:2454] T 87769642c2964fba856bf07a2dfab9df P 16db83d271ac4c2fa0f13fae0e6339d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bd444f3769b4452a20f74b5341c1ac2 in term 1.
I20260809 20:20:37.239111 21439 leader_election.cc:302] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bd444f3769b4452a20f74b5341c1ac2, 8e6be863daab4c0da5d4e268a24f4ca0; no voters:
I20260809 20:20:37.239424 21731 raft_consensus.cc:2781] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:20:37.239890 21731 raft_consensus.cc:686] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2 [term 1 LEADER]: Becoming Leader. State: Replica: 3bd444f3769b4452a20f74b5341c1ac2, State: Running, Role: LEADER
I20260809 20:20:37.240336 21731 consensus_queue.cc:237] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6be863daab4c0da5d4e268a24f4ca0" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46119 } } peers { permanent_uuid: "3bd444f3769b4452a20f74b5341c1ac2" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34751 } } peers { permanent_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 45771 } }
I20260809 20:20:37.244769 21207 catalog_manager.cc:5495] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bd444f3769b4452a20f74b5341c1ac2 (127.5.61.2). New cstate: current_term: 1 leader_uuid: "3bd444f3769b4452a20f74b5341c1ac2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6be863daab4c0da5d4e268a24f4ca0" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bd444f3769b4452a20f74b5341c1ac2" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 45771 } health_report { overall_health: UNKNOWN } } }
I20260809 20:20:37.268187 21716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:37.262469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47882 (local address 127.5.61.2:34751)
0809 20:20:37.262689 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:37.262695 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:37.262713 (+ 18us) server_negotiation.cc:407] Connection header received
0809 20:20:37.262848 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:37.262852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:37.262964 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:37.263118 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:20:37.263549 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.264452 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.265982 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.266380 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.266513 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:37.267049 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:20:37.267525 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:20:37.267742 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:37.267797 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 20:20:37.267893 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 20:20:37.277056 21363 raft_consensus.cc:1261] T 87769642c2964fba856bf07a2dfab9df P 16db83d271ac4c2fa0f13fae0e6339d7 [term 1 FOLLOWER]: Refusing update from remote peer 3bd444f3769b4452a20f74b5341c1ac2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:20:37.277088 21655 raft_consensus.cc:1261] T 87769642c2964fba856bf07a2dfab9df P 8e6be863daab4c0da5d4e268a24f4ca0 [term 1 FOLLOWER]: Refusing update from remote peer 3bd444f3769b4452a20f74b5341c1ac2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:20:37.278008 21734 consensus_queue.cc:1035] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 45771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:20:37.278348 21731 consensus_queue.cc:1035] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e6be863daab4c0da5d4e268a24f4ca0" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 20:20:37.280530 21415 tablet.cc:2369] T 87769642c2964fba856bf07a2dfab9df P 16db83d271ac4c2fa0f13fae0e6339d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:20:37.290376 21738 mvcc.cc:205] Tried to move back new op lower bound from 7316712805476651008 to 7316712805339643904. Current Snapshot: MvccSnapshot[applied={T|T < 7316712805476651008}]
I20260809 20:20:37.303328 21207 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:20:37.303642 21207 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:20:37.306869 21207 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477: Sending DeleteTablet for 3 replicas of tablet 87769642c2964fba856bf07a2dfab9df
I20260809 20:20:37.308022 21633 tablet_service.cc:1514] Processing DeleteTablet for tablet 87769642c2964fba856bf07a2dfab9df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:20:37 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:55806
I20260809 20:20:37.308193 21491 tablet_service.cc:1514] Processing DeleteTablet for tablet 87769642c2964fba856bf07a2dfab9df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:20:37 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:47872
I20260809 20:20:37.308283 21343 tablet_service.cc:1514] Processing DeleteTablet for tablet 87769642c2964fba856bf07a2dfab9df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:20:37 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:57062
I20260809 20:20:37.309065 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 21273
I20260809 20:20:37.335151 21750 tablet_replica.cc:330] T 87769642c2964fba856bf07a2dfab9df P 16db83d271ac4c2fa0f13fae0e6339d7: stopping tablet replica
I20260809 20:20:37.335781 21750 raft_consensus.cc:2229] T 87769642c2964fba856bf07a2dfab9df P 16db83d271ac4c2fa0f13fae0e6339d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:20:37.336411 21748 tablet_replica.cc:330] T 87769642c2964fba856bf07a2dfab9df P 8e6be863daab4c0da5d4e268a24f4ca0: stopping tablet replica
I20260809 20:20:37.337003 21750 raft_consensus.cc:2258] T 87769642c2964fba856bf07a2dfab9df P 16db83d271ac4c2fa0f13fae0e6339d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:20:37.337805 21748 raft_consensus.cc:2229] T 87769642c2964fba856bf07a2dfab9df P 8e6be863daab4c0da5d4e268a24f4ca0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:20:37.338325 21748 raft_consensus.cc:2258] T 87769642c2964fba856bf07a2dfab9df P 8e6be863daab4c0da5d4e268a24f4ca0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:20:37.339171 21749 tablet_replica.cc:330] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2: stopping tablet replica
I20260809 20:20:37.339612 21749 raft_consensus.cc:2229] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:20:37.340389 21749 raft_consensus.cc:2258] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:20:37.340794 21748 ts_tablet_manager.cc:1905] T 87769642c2964fba856bf07a2dfab9df P 8e6be863daab4c0da5d4e268a24f4ca0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:20:37.340808 21750 ts_tablet_manager.cc:1905] T 87769642c2964fba856bf07a2dfab9df P 16db83d271ac4c2fa0f13fae0e6339d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:20:37.345335 21749 ts_tablet_manager.cc:1905] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:20:37.346838 21748 ts_tablet_manager.cc:1918] T 87769642c2964fba856bf07a2dfab9df P 8e6be863daab4c0da5d4e268a24f4ca0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:20:37.347033 21748 log.cc:1196] T 87769642c2964fba856bf07a2dfab9df P 8e6be863daab4c0da5d4e268a24f4ca0: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/87769642c2964fba856bf07a2dfab9df
I20260809 20:20:37.347617 21748 ts_tablet_manager.cc:1939] T 87769642c2964fba856bf07a2dfab9df P 8e6be863daab4c0da5d4e268a24f4ca0: Deleting consensus metadata
I20260809 20:20:37.347999 21750 ts_tablet_manager.cc:1918] T 87769642c2964fba856bf07a2dfab9df P 16db83d271ac4c2fa0f13fae0e6339d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:20:37.348179 21750 log.cc:1196] T 87769642c2964fba856bf07a2dfab9df P 16db83d271ac4c2fa0f13fae0e6339d7: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/87769642c2964fba856bf07a2dfab9df
I20260809 20:20:37.348665 21750 ts_tablet_manager.cc:1939] T 87769642c2964fba856bf07a2dfab9df P 16db83d271ac4c2fa0f13fae0e6339d7: Deleting consensus metadata
I20260809 20:20:37.349468 21190 catalog_manager.cc:4841] TS 8e6be863daab4c0da5d4e268a24f4ca0 (127.5.61.3:46119): tablet 87769642c2964fba856bf07a2dfab9df (table test-table [id=800e458a01d04a5f8fbbc23fb3bf6310]) successfully deleted
I20260809 20:20:37.350286 21193 catalog_manager.cc:4841] TS 16db83d271ac4c2fa0f13fae0e6339d7 (127.5.61.1:45771): tablet 87769642c2964fba856bf07a2dfab9df (table test-table [id=800e458a01d04a5f8fbbc23fb3bf6310]) successfully deleted
I20260809 20:20:37.352411 21749 ts_tablet_manager.cc:1918] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:20:37.352568 21749 log.cc:1196] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/87769642c2964fba856bf07a2dfab9df
I20260809 20:20:37.353084 21749 ts_tablet_manager.cc:1939] T 87769642c2964fba856bf07a2dfab9df P 3bd444f3769b4452a20f74b5341c1ac2: Deleting consensus metadata
Aug 09 20:20:37 dist-test-slave-6k22 krb5kdc[21155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.1@KRBTEST.COM
I20260809 20:20:37.354720 21193 catalog_manager.cc:4841] TS 3bd444f3769b4452a20f74b5341c1ac2 (127.5.61.2:34751): tablet 87769642c2964fba856bf07a2dfab9df (table test-table [id=800e458a01d04a5f8fbbc23fb3bf6310]) successfully deleted
I20260809 20:20:37.361011 21719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:37.310205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57068 (local address 127.5.61.1:45771)
0809 20:20:37.310440 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:37.310448 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:37.334292 (+ 23844us) server_negotiation.cc:407] Connection header received
0809 20:20:37.335977 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:37.335987 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:37.336114 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:37.336257 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:37.339687 (+ 3430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.340765 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.342990 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.343740 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.343832 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:37.355353 (+ 11521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:37.355365 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:37.355377 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:37.355413 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:37.358118 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:37.358830 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:37.358837 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:37.358839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:37.358967 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:37.359356 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:37.359365 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:37.359367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:37.359797 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:37.360099 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:37.360417 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 20:20:37.360629 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 20:20:37.361585 21409 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:20:37.485069 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 21273
I20260809 20:20:37.506731 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 21421
Aug 09 20:20:37 dist-test-slave-6k22 krb5kdc[21155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.2@KRBTEST.COM
I20260809 20:20:37.525295 21716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:37.507662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47898 (local address 127.5.61.2:34751)
0809 20:20:37.507862 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:37.507868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:37.507978 (+ 110us) server_negotiation.cc:407] Connection header received
0809 20:20:37.508215 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:37.508222 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:37.508316 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:37.508459 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:37.510173 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.511095 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.513153 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.513573 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.513668 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:37.518859 (+ 5191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:37.518870 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:37.518885 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:37.518920 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:37.522111 (+ 3191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:37.522820 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:37.522827 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:37.522829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:37.522976 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:37.523399 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:37.523405 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:37.523410 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:37.524007 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:37.524392 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:37.524629 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 20:20:37.524887 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 20:20:37.526042 21554 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:20:37.638628 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 21421
I20260809 20:20:37.662395 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 21566
Aug 09 20:20:37 dist-test-slave-6k22 krb5kdc[21155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.3@KRBTEST.COM
I20260809 20:20:37.676752 21717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:37.663208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55810 (local address 127.5.61.3:46119)
0809 20:20:37.663357 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:37.663369 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:37.663405 (+ 36us) server_negotiation.cc:407] Connection header received
0809 20:20:37.663640 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:37.663644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:37.663709 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:37.663789 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:37.665155 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.665888 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.667075 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:37.667461 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:37.667604 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:37.671798 (+ 4194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:37.671812 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:37.671827 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:37.671863 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:37.674472 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:37.675059 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:37.675066 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:37.675068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:37.675200 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:37.675495 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:37.675498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:37.675500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:37.675883 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:37.676093 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:37.676351 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 20:20:37.676514 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 20:20:37.677412 21696 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:20:37.766407 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 21566
I20260809 20:20:37.787984 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 21169
I20260809 20:20:37.789194 21240 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:20:37.895912 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 21169
I20260809 20:20:37.920257 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:46463
--webserver_interface=127.5.61.62
--webserver_port=35987
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:32969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:46463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:20:38.255940 21756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:38.256219 21756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:38.256448 21756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:20:38.263722 21756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:20:38.263852 21756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:38.263932 21756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:38.264037 21756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:20:38.264109 21756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:20:38.274832 21756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:32969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:46463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:46463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=35987
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:38.276302 21756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:38.277594 21756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:38.301430 21764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:38.301405 21768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:38.301378 21765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:38.302350 21756 server_base.cc:1034] running on GCE node
I20260809 20:20:38.303184 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:38.304832 21756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:38.306041 21756 hybrid_clock.cc:648] HybridClock initialized: now 1786306838305998 us; error 39 us; skew 500 ppm
I20260809 20:20:38.306448 21756 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:20:38 dist-test-slave-6k22 krb5kdc[21155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306838, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:38.310299 21756 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:20:38.312189 21756 webserver.cc:460] Webserver started at http://127.5.61.62:35987/ using document root <none> and password file <none>
I20260809 20:20:38.312896 21756 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:38.313019 21756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:20:38.318827 21756 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 20:20:38.321662 21779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:38.323005 21756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 20:20:38.323143 21756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "8c69b8e666d245ada106cc1db1ac8477"
format_stamp: "Formatted at 2026-08-09 20:20:35 on dist-test-slave-6k22"
server_key: "2e07a2de0c7414fd9eb5f6683382fc64"
server_key_iv: "c1133fc7cd642091af6bf4a9ec263ce4"
server_key_version: "encryptionkey@0"
I20260809 20:20:38.324260 21756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:38.348742 21756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:38.352744 21756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:38.352960 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:38.372051 21756 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:46463
I20260809 20:20:38.372068 21836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:46463 every 8 connection(s)
I20260809 20:20:38.373581 21756 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:20:38.375861 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 21756
I20260809 20:20:38.376729 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:45771
--local_ip_for_outbound_sockets=127.5.61.1
--tserver_master_addrs=127.5.61.62:46463
--webserver_port=41785
--webserver_interface=127.5.61.1
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:32969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 20:20:38.380975 21837 sys_catalog.cc:263] Verifying existing consensus state
I20260809 20:20:38.385311 21837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477: Bootstrap starting.
I20260809 20:20:38.404840 21837 log.cc:824] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477: Log is configured to *not* fsync() on all Append() calls
I20260809 20:20:38.417575 21837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 20:20:38.418159 21837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477: Bootstrap complete.
I20260809 20:20:38.427222 21837 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c69b8e666d245ada106cc1db1ac8477" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46463 } }
I20260809 20:20:38.428244 21837 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c69b8e666d245ada106cc1db1ac8477, State: Initialized, Role: FOLLOWER
I20260809 20:20:38.428953 21837 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c69b8e666d245ada106cc1db1ac8477" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46463 } }
I20260809 20:20:38.429270 21837 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:20:38.429395 21837 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:20:38.429598 21837 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 1 FOLLOWER]: Advancing to term 2
I20260809 20:20:38.433350 21837 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c69b8e666d245ada106cc1db1ac8477" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46463 } }
I20260809 20:20:38.433930 21837 leader_election.cc:302] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c69b8e666d245ada106cc1db1ac8477; no voters:
I20260809 20:20:38.434602 21837 leader_election.cc:288] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 20:20:38.434751 21843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 20:20:38.436079 21843 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 2 LEADER]: Becoming Leader. State: Replica: 8c69b8e666d245ada106cc1db1ac8477, State: Running, Role: LEADER
I20260809 20:20:38.436633 21843 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c69b8e666d245ada106cc1db1ac8477" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46463 } }
I20260809 20:20:38.437062 21837 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:20:38.439201 21845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c69b8e666d245ada106cc1db1ac8477. Latest consensus state: current_term: 2 leader_uuid: "8c69b8e666d245ada106cc1db1ac8477" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c69b8e666d245ada106cc1db1ac8477" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46463 } } }
I20260809 20:20:38.440208 21845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [sys.catalog]: This master's current role is: LEADER
I20260809 20:20:38.439854 21844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8c69b8e666d245ada106cc1db1ac8477" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c69b8e666d245ada106cc1db1ac8477" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46463 } } }
I20260809 20:20:38.440646 21844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [sys.catalog]: This master's current role is: LEADER
I20260809 20:20:38.444082 21849 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:20:38.454016 21849 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:20:38.455533 21849 catalog_manager.cc:1259] Loaded cluster ID: 84b3bfbcdc594f2db1e6498ba6dfebc9
I20260809 20:20:38.455720 21849 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:20:38.462749 21849 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 20:20:38.480964 21849 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 20:20:38.501351 21849 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:20:38.503199 21849 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477: Loaded TSK: 0
I20260809 20:20:38.504719 21849 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 20:20:38.733548 21841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:38.733887 21841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:38.734056 21841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:20:38.741573 21841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:38.741741 21841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:38.741921 21841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:20:38.751755 21841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:32969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:45771
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=41785
--tserver_master_addrs=127.5.61.62:46463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:38.753048 21841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:38.754344 21841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:38.769438 21870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:38.769446 21869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:38.770936 21872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:38.771126 21841 server_base.cc:1034] running on GCE node
I20260809 20:20:38.771780 21841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:38.772819 21841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:38.774060 21841 hybrid_clock.cc:648] HybridClock initialized: now 1786306838774007 us; error 47 us; skew 500 ppm
I20260809 20:20:38.774389 21841 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:20:38 dist-test-slave-6k22 krb5kdc[21155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306838, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:38.781978 21841 init.cc:377] Logged in from keytab as kudu/127.5.61.1@KRBTEST.COM (short username kudu)
I20260809 20:20:38.783665 21841 webserver.cc:460] Webserver started at http://127.5.61.1:41785/ using document root <none> and password file <none>
I20260809 20:20:38.784356 21841 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:38.784509 21841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:20:38.789923 21841 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 20:20:38.792717 21882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:38.794101 21841 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 20:20:38.794272 21841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "16db83d271ac4c2fa0f13fae0e6339d7"
format_stamp: "Formatted at 2026-08-09 20:20:35 on dist-test-slave-6k22"
server_key: "410044219cbfc5ecfd1e5611a15035d6"
server_key_iv: "fb6ffcf585f70f4358959337f2bcbeaa"
server_key_version: "encryptionkey@0"
I20260809 20:20:38.795236 21841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:38.811172 21841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:38.815407 21841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:38.815686 21841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:38.816754 21841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:20:38.818190 21841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:20:38.818279 21841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:38.818446 21841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:20:38.818547 21841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:38.856426 21841 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:45771
I20260809 20:20:38.856509 22007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:45771 every 8 connection(s)
I20260809 20:20:38.858337 21841 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:20:38.863550 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 21841
I20260809 20:20:38.864413 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:34751
--local_ip_for_outbound_sockets=127.5.61.2
--tserver_master_addrs=127.5.61.62:46463
--webserver_port=40065
--webserver_interface=127.5.61.2
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:32969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 20:20:38 dist-test-slave-6k22 krb5kdc[21155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306838, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:20:38.882251 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:38.861078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:46687 (local address 127.5.61.62:46463)
0809 20:20:38.861759 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:38.861774 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:38.862153 (+ 379us) server_negotiation.cc:407] Connection header received
0809 20:20:38.864327 (+ 2174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:38.864346 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:38.864798 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:38.865191 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:38.866535 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:38.868552 (+ 2017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:38.870029 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:38.871142 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:38.871327 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:38.874678 (+ 3351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:38.874685 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:38.874698 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:38.874738 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:38.877580 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:38.878556 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:38.878562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:38.878567 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:38.878715 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:38.879174 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:38.879177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:38.879179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:38.879681 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:38.879901 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:38.880653 (+ 752us) server_negotiation.cc:299] Negotiation successful
0809 20:20:38.881036 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":251,"threads_started":1}
I20260809 20:20:38.882249 22011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:38.861210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.1:46687)
0809 20:20:38.861979 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:38.862023 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:38.863741 (+ 1718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:38.865302 (+ 1561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:38.865309 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:38.865771 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:20:38.866290 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:38.866306 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:38.868702 (+ 2396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:38.868706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:38.869877 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:38.869888 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:38.871352 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:38.871356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:38.871532 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:38.872269 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:20:38.872296 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:20:38.874397 (+ 2101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:20:38.877724 (+ 3327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:38.877734 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:38.877750 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:38.878430 (+ 680us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:38.878824 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:38.878831 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:38.878837 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:38.879069 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:38.879825 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:20:38.879835 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:20:38.880168 (+ 333us) client_negotiation.cc:769] Sending connection context
0809 20:20:38.880441 (+ 273us) client_negotiation.cc:240] Negotiation successful
0809 20:20:38.880951 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":302,"threads_started":1}
I20260809 20:20:38.885469 22008 heartbeater.cc:347] Connected to a master server at 127.5.61.62:46463
I20260809 20:20:38.885785 22008 heartbeater.cc:464] Registering TS with master...
I20260809 20:20:38.886615 22008 heartbeater.cc:511] Master 127.5.61.62:46463 requested a full tablet report, sending...
I20260809 20:20:38.888813 21796 ts_manager.cc:194] Registered new tserver with Master: 16db83d271ac4c2fa0f13fae0e6339d7 (127.5.61.1:45771)
I20260809 20:20:38.890201 21796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.1@KRBTEST.COM'} at 127.5.61.1:46687
W20260809 20:20:39.217221 22013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:39.217542 22013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:39.217734 22013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:20:39.224911 22013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:39.225069 22013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:39.225247 22013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:20:39.235033 22013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:32969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:34751
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=40065
--tserver_master_addrs=127.5.61.62:46463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:39.236342 22013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:39.237794 22013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:39.251288 22019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:39.251358 22018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:39.251282 22021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:39.252022 22013 server_base.cc:1034] running on GCE node
I20260809 20:20:39.252703 22013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:39.253569 22013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:39.254841 22013 hybrid_clock.cc:648] HybridClock initialized: now 1786306839254789 us; error 36 us; skew 500 ppm
I20260809 20:20:39.255175 22013 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:20:39 dist-test-slave-6k22 krb5kdc[21155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306839, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:39.258426 22013 init.cc:377] Logged in from keytab as kudu/127.5.61.2@KRBTEST.COM (short username kudu)
I20260809 20:20:39.259930 22013 webserver.cc:460] Webserver started at http://127.5.61.2:40065/ using document root <none> and password file <none>
I20260809 20:20:39.260578 22013 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:39.260740 22013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:20:39.265863 22013 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 20:20:39.268530 22028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:39.269771 22013 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 20:20:39.269942 22013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "3bd444f3769b4452a20f74b5341c1ac2"
format_stamp: "Formatted at 2026-08-09 20:20:36 on dist-test-slave-6k22"
server_key: "53ad4d1cac2ccc6db4b91968915fb390"
server_key_iv: "e252fc5413065112e812f8de2bc48976"
server_key_version: "encryptionkey@0"
I20260809 20:20:39.270947 22013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:39.287256 22013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:39.292205 22013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:39.292505 22013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:39.293453 22013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:20:39.294915 22013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:20:39.295055 22013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:39.295188 22013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:20:39.295289 22013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:39.332935 22013 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:34751
I20260809 20:20:39.333031 22151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:34751 every 8 connection(s)
I20260809 20:20:39.334560 22013 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:20:39.343833 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 22013
I20260809 20:20:39.344609 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:46119
--local_ip_for_outbound_sockets=127.5.61.3
--tserver_master_addrs=127.5.61.62:46463
--webserver_port=38121
--webserver_interface=127.5.61.3
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:32969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 20:20:39 dist-test-slave-6k22 krb5kdc[21155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306839, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:20:39.356419 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:39.337252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:58519 (local address 127.5.61.62:46463)
0809 20:20:39.337469 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:39.337474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:39.338694 (+ 1220us) server_negotiation.cc:407] Connection header received
0809 20:20:39.340608 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:39.340614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:39.340718 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:39.340848 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:39.342457 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:39.343131 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:39.345973 (+ 2842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:39.346414 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:39.346512 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:39.350910 (+ 4398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:39.350914 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:39.350917 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:39.350949 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:39.353414 (+ 2465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:39.354134 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:39.354144 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:39.354146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:39.354266 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:39.354706 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:39.354711 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:39.354715 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:39.354977 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:39.355132 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:39.355843 (+ 711us) server_negotiation.cc:299] Negotiation successful
0809 20:20:39.356077 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260809 20:20:39.357487 22155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:39.337713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.2:58519)
0809 20:20:39.338530 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:39.338576 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:39.340341 (+ 1765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:39.341089 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:39.341097 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:39.341595 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:20:39.342254 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:39.342282 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:39.343349 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:39.343356 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:39.345789 (+ 2433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:39.345808 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:39.346664 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:39.346674 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:39.347028 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:39.347873 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:20:39.347910 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:20:39.350698 (+ 2788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:20:39.353564 (+ 2866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:39.353572 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:39.353585 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:39.354000 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:39.354386 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:39.354390 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:39.354394 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:39.354583 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:39.355106 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:20:39.355114 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:20:39.355436 (+ 322us) client_negotiation.cc:769] Sending connection context
0809 20:20:39.355730 (+ 294us) client_negotiation.cc:240] Negotiation successful
0809 20:20:39.356229 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":294,"threads_started":1}
I20260809 20:20:39.358996 22152 heartbeater.cc:347] Connected to a master server at 127.5.61.62:46463
I20260809 20:20:39.359308 22152 heartbeater.cc:464] Registering TS with master...
I20260809 20:20:39.360158 22152 heartbeater.cc:511] Master 127.5.61.62:46463 requested a full tablet report, sending...
I20260809 20:20:39.361775 21796 ts_manager.cc:194] Registered new tserver with Master: 3bd444f3769b4452a20f74b5341c1ac2 (127.5.61.2:34751)
I20260809 20:20:39.363260 21796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.2@KRBTEST.COM'} at 127.5.61.2:58519
W20260809 20:20:39.686491 22157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:39.686754 22157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:39.686849 22157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:20:39.693743 22157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:39.693842 22157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:39.693984 22157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:20:39.703615 22157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:32969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:46119
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=38121
--tserver_master_addrs=127.5.61.62:46463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:39.704802 22157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:39.706071 22157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:39.719492 22162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:39.719465 22163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:39.719571 22165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:39.720211 22157 server_base.cc:1034] running on GCE node
I20260809 20:20:39.720808 22157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:39.721668 22157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:39.722889 22157 hybrid_clock.cc:648] HybridClock initialized: now 1786306839722849 us; error 56 us; skew 500 ppm
I20260809 20:20:39.723217 22157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:20:39 dist-test-slave-6k22 krb5kdc[21155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306839, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:39.726228 22157 init.cc:377] Logged in from keytab as kudu/127.5.61.3@KRBTEST.COM (short username kudu)
I20260809 20:20:39.727698 22157 webserver.cc:460] Webserver started at http://127.5.61.3:38121/ using document root <none> and password file <none>
I20260809 20:20:39.728353 22157 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:39.728505 22157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:20:39.733640 22157 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 20:20:39.736212 22172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:39.737478 22157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:20:39.737645 22157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "8e6be863daab4c0da5d4e268a24f4ca0"
format_stamp: "Formatted at 2026-08-09 20:20:37 on dist-test-slave-6k22"
server_key: "4f5abad1086689b31f9c9c9beac7a159"
server_key_iv: "a9e34eea7de7dd903398d049a504b575"
server_key_version: "encryptionkey@0"
I20260809 20:20:39.738656 22157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:39.760206 22157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:39.763810 22157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:39.764067 22157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:39.764984 22157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:20:39.766245 22157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:20:39.766371 22157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:39.766494 22157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:20:39.766579 22157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:39.799643 22157 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:46119
I20260809 20:20:39.799739 22300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:46119 every 8 connection(s)
I20260809 20:20:39.801079 22157 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:20:39.804767 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 22157
Aug 09 20:20:39 dist-test-slave-6k22 krb5kdc[21155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306839, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:20:39.823969 22305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:39.805836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59760 (local address 127.5.61.62:46463)
0809 20:20:39.807951 (+ 2115us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:39.807956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:39.807969 (+ 13us) server_negotiation.cc:407] Connection header received
0809 20:20:39.808011 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:39.808014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:39.808061 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:39.808147 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:39.810209 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:39.811257 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:39.813151 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:39.814001 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:39.814188 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:39.817547 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:39.817551 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:39.817554 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:39.817583 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:39.820882 (+ 3299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:39.821709 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:39.821716 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:39.821719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:39.821861 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:39.822311 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:39.822316 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:39.822320 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:39.822701 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:39.822943 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:39.823314 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 20:20:39.823529 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":148,"server-negotiator.queue_time_us":2029,"spinlock_wait_cycles":28160,"thread_start_us":1900,"threads_started":1}
I20260809 20:20:39.824625 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:39.803506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:49139 (local address 127.5.61.62:46463)
0809 20:20:39.803701 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:39.803705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:39.804975 (+ 1270us) server_negotiation.cc:407] Connection header received
0809 20:20:39.807500 (+ 2525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:39.807506 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:39.807634 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:39.807773 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:39.809871 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:39.810989 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:39.813151 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:39.814001 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:39.814188 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:39.818220 (+ 4032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:39.818225 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:39.818228 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:39.818257 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:39.821201 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:39.822007 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:39.822011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:39.822015 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:39.822149 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:39.822698 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:39.822701 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:39.822703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:39.823103 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:39.823242 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:39.824129 (+ 887us) server_negotiation.cc:299] Negotiation successful
0809 20:20:39.824318 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":185,"server-negotiator.queue_time_us":74,"spinlock_wait_cycles":6016}
I20260809 20:20:39.825948 22303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:39.803920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.3:49139)
0809 20:20:39.804782 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:39.804833 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:39.807169 (+ 2336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:39.808079 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:39.808088 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:39.808852 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:20:39.809689 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:39.809715 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:39.811148 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:39.811151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:39.813025 (+ 1874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:39.813040 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:39.814246 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:39.814253 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:39.814554 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:39.815296 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:20:39.815330 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:20:39.818010 (+ 2680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:20:39.821332 (+ 3322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:39.821338 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:39.821350 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:39.821884 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:39.822311 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:20:39.822316 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:20:39.822320 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:20:39.822580 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:20:39.823240 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:20:39.823248 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:20:39.823681 (+ 433us) client_negotiation.cc:769] Sending connection context
0809 20:20:39.823999 (+ 318us) client_negotiation.cc:240] Negotiation successful
0809 20:20:39.824869 (+ 870us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":327,"threads_started":1}
I20260809 20:20:39.827334 22301 heartbeater.cc:347] Connected to a master server at 127.5.61.62:46463
I20260809 20:20:39.827698 22301 heartbeater.cc:464] Registering TS with master...
I20260809 20:20:39.828416 22301 heartbeater.cc:511] Master 127.5.61.62:46463 requested a full tablet report, sending...
I20260809 20:20:39.829802 21796 ts_manager.cc:194] Registered new tserver with Master: 8e6be863daab4c0da5d4e268a24f4ca0 (127.5.61.3:46119)
I20260809 20:20:39.830559 21796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.3@KRBTEST.COM'} at 127.5.61.3:49139
I20260809 20:20:39.838933 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 20:20:39.845444 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:39.839802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59774 (local address 127.5.61.62:46463)
0809 20:20:39.839956 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:39.839960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:39.840443 (+ 483us) server_negotiation.cc:407] Connection header received
0809 20:20:39.840599 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:39.840602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:39.840749 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:39.840865 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:20:39.841240 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:39.841960 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:39.843438 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:39.843954 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:39.844046 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:39.844497 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:20:39.844966 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:20:39.845087 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:39.845188 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 20:20:39.845248 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 20:20:39.846330 21796 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:59774:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:20:39.853595 21796 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:20:39.880463 22312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:39.865773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:34751 (local address 127.0.0.1:49640)
0809 20:20:39.867664 (+ 1891us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:39.867714 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:39.867929 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:39.870101 (+ 2172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:39.870112 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:39.870156 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:20:39.870693 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:39.870709 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:39.873581 (+ 2872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:39.873590 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:39.877104 (+ 3514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:39.877134 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:39.879343 (+ 2209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:39.879350 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:39.879609 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:39.879673 (+ 64us) client_negotiation.cc:769] Sending connection context
0809 20:20:39.879806 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 20:20:39.879989 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1542,"mutex_wait_us":110,"thread_start_us":3498,"threads_started":1}
I20260809 20:20:39.881250 22314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:39.866499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38082 (local address 127.5.61.3:46119)
0809 20:20:39.868799 (+ 2300us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:39.868811 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:39.869856 (+ 1045us) server_negotiation.cc:407] Connection header received
0809 20:20:39.869979 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:39.869987 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:39.870207 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:39.870464 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:20:39.871812 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:39.874166 (+ 2354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:39.878306 (+ 4140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:39.879794 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:39.879921 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:39.880133 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:39.880744 (+ 611us) server_negotiation.cc:299] Negotiation successful
0809 20:20:39.880955 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2075,"thread_start_us":1811,"threads_started":1}
I20260809 20:20:39.882086 22311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:39.866034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:46119 (local address 127.0.0.1:38082)
0809 20:20:39.869503 (+ 3469us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:39.869520 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:39.869698 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:39.870859 (+ 1161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:39.870867 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:39.870906 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:20:39.871458 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:39.871474 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:39.874476 (+ 3002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:39.874486 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:39.878118 (+ 3632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:39.878141 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:39.880302 (+ 2161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:39.880308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:39.880516 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:39.880553 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 20:20:39.881513 (+ 960us) client_negotiation.cc:240] Negotiation successful
0809 20:20:39.881654 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3366,"mutex_wait_us":473,"thread_start_us":1115,"threads_started":1}
I20260809 20:20:39.882274 22313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:39.866004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49640 (local address 127.5.61.2:34751)
0809 20:20:39.869332 (+ 3328us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:39.869343 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:39.869382 (+ 39us) server_negotiation.cc:407] Connection header received
0809 20:20:39.869493 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:39.869504 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:39.869783 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:39.870930 (+ 1147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:20:39.871224 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:39.873372 (+ 2148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:39.877400 (+ 4028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:39.879090 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:39.880956 (+ 1866us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:39.881132 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:39.881591 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 20:20:39.881766 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3144,"thread_start_us":2966,"threads_started":1}
I20260809 20:20:39.885887 22081 tablet_service.cc:1467] Processing CreateTablet for tablet 8ed86ca2f5584c3eaf73778c56822fa5 (DEFAULT_TABLE table=test-table [id=b37a29f863ae4887bdfc6496d5d09ae2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:20:39.885891 22224 tablet_service.cc:1467] Processing CreateTablet for tablet 8ed86ca2f5584c3eaf73778c56822fa5 (DEFAULT_TABLE table=test-table [id=b37a29f863ae4887bdfc6496d5d09ae2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:20:39.886572 22315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:39.869654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:45771 (local address 127.0.0.1:53416)
0809 20:20:39.877897 (+ 8243us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:39.877917 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:39.878480 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:39.879155 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:39.879160 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:39.879188 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:20:39.879738 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:39.879749 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:39.881485 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:39.881491 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:39.883869 (+ 2378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:39.883889 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:39.885738 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:39.885745 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:39.885954 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:39.885991 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 20:20:39.886087 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 20:20:39.886230 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8097,"thread_start_us":7920,"threads_started":1}
I20260809 20:20:39.886617 22316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:39.870002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53416 (local address 127.5.61.1:45771)
0809 20:20:39.877390 (+ 7388us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:39.877399 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:39.878429 (+ 1030us) server_negotiation.cc:407] Connection header received
0809 20:20:39.878664 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:39.878676 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:39.878937 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:39.879179 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:20:39.879910 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:39.881287 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:39.884078 (+ 2791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:39.885493 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:39.885628 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:39.885836 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:39.886164 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 20:20:39.886311 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7224,"thread_start_us":7062,"threads_started":1}
I20260809 20:20:39.887439 22081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ed86ca2f5584c3eaf73778c56822fa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:20:39.887442 22224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ed86ca2f5584c3eaf73778c56822fa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:20:39.890795 21935 tablet_service.cc:1467] Processing CreateTablet for tablet 8ed86ca2f5584c3eaf73778c56822fa5 (DEFAULT_TABLE table=test-table [id=b37a29f863ae4887bdfc6496d5d09ae2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:20:39.892625 21935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ed86ca2f5584c3eaf73778c56822fa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:20:39.892973 22008 heartbeater.cc:503] Master 127.5.61.62:46463 was elected leader, sending a full tablet report...
I20260809 20:20:39.900838 22317 tablet_bootstrap.cc:492] T 8ed86ca2f5584c3eaf73778c56822fa5 P 8e6be863daab4c0da5d4e268a24f4ca0: Bootstrap starting.
I20260809 20:20:39.903898 22318 tablet_bootstrap.cc:492] T 8ed86ca2f5584c3eaf73778c56822fa5 P 3bd444f3769b4452a20f74b5341c1ac2: Bootstrap starting.
I20260809 20:20:39.904528 22317 tablet_bootstrap.cc:654] T 8ed86ca2f5584c3eaf73778c56822fa5 P 8e6be863daab4c0da5d4e268a24f4ca0: Neither blocks nor log segments found. Creating new log.
I20260809 20:20:39.905957 22317 log.cc:824] T 8ed86ca2f5584c3eaf73778c56822fa5 P 8e6be863daab4c0da5d4e268a24f4ca0: Log is configured to *not* fsync() on all Append() calls
I20260809 20:20:39.907615 22318 tablet_bootstrap.cc:654] T 8ed86ca2f5584c3eaf73778c56822fa5 P 3bd444f3769b4452a20f74b5341c1ac2: Neither blocks nor log segments found. Creating new log.
I20260809 20:20:39.908888 22318 log.cc:824] T 8ed86ca2f5584c3eaf73778c56822fa5 P 3bd444f3769b4452a20f74b5341c1ac2: Log is configured to *not* fsync() on all Append() calls
I20260809 20:20:39.909384 22317 tablet_bootstrap.cc:492] T 8ed86ca2f5584c3eaf73778c56822fa5 P 8e6be863daab4c0da5d4e268a24f4ca0: No bootstrap required, opened a new log
I20260809 20:20:39.910162 22317 ts_tablet_manager.cc:1397] T 8ed86ca2f5584c3eaf73778c56822fa5 P 8e6be863daab4c0da5d4e268a24f4ca0: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 20:20:39.910343 22321 tablet_bootstrap.cc:492] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7: Bootstrap starting.
I20260809 20:20:39.912282 22318 tablet_bootstrap.cc:492] T 8ed86ca2f5584c3eaf73778c56822fa5 P 3bd444f3769b4452a20f74b5341c1ac2: No bootstrap required, opened a new log
I20260809 20:20:39.912569 22318 ts_tablet_manager.cc:1397] T 8ed86ca2f5584c3eaf73778c56822fa5 P 3bd444f3769b4452a20f74b5341c1ac2: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260809 20:20:39.914003 22321 tablet_bootstrap.cc:654] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7: Neither blocks nor log segments found. Creating new log.
I20260809 20:20:39.915311 22321 log.cc:824] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7: Log is configured to *not* fsync() on all Append() calls
I20260809 20:20:39.918494 22321 tablet_bootstrap.cc:492] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7: No bootstrap required, opened a new log
I20260809 20:20:39.918802 22321 ts_tablet_manager.cc:1397] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 20:20:39.921418 22318 raft_consensus.cc:348] T 8ed86ca2f5584c3eaf73778c56822fa5 P 3bd444f3769b4452a20f74b5341c1ac2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd444f3769b4452a20f74b5341c1ac2" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34751 } } peers { permanent_uuid: "8e6be863daab4c0da5d4e268a24f4ca0" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46119 } } peers { permanent_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 45771 } }
I20260809 20:20:39.921512 22317 raft_consensus.cc:348] T 8ed86ca2f5584c3eaf73778c56822fa5 P 8e6be863daab4c0da5d4e268a24f4ca0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd444f3769b4452a20f74b5341c1ac2" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34751 } } peers { permanent_uuid: "8e6be863daab4c0da5d4e268a24f4ca0" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46119 } } peers { permanent_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 45771 } }
I20260809 20:20:39.921816 22318 raft_consensus.cc:374] T 8ed86ca2f5584c3eaf73778c56822fa5 P 3bd444f3769b4452a20f74b5341c1ac2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:20:39.921936 22318 raft_consensus.cc:729] T 8ed86ca2f5584c3eaf73778c56822fa5 P 3bd444f3769b4452a20f74b5341c1ac2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd444f3769b4452a20f74b5341c1ac2, State: Initialized, Role: FOLLOWER
I20260809 20:20:39.921989 22317 raft_consensus.cc:374] T 8ed86ca2f5584c3eaf73778c56822fa5 P 8e6be863daab4c0da5d4e268a24f4ca0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:20:39.922091 22317 raft_consensus.cc:729] T 8ed86ca2f5584c3eaf73778c56822fa5 P 8e6be863daab4c0da5d4e268a24f4ca0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e6be863daab4c0da5d4e268a24f4ca0, State: Initialized, Role: FOLLOWER
I20260809 20:20:39.922711 22318 consensus_queue.cc:260] T 8ed86ca2f5584c3eaf73778c56822fa5 P 3bd444f3769b4452a20f74b5341c1ac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd444f3769b4452a20f74b5341c1ac2" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34751 } } peers { permanent_uuid: "8e6be863daab4c0da5d4e268a24f4ca0" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46119 } } peers { permanent_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 45771 } }
I20260809 20:20:39.922686 22317 consensus_queue.cc:260] T 8ed86ca2f5584c3eaf73778c56822fa5 P 8e6be863daab4c0da5d4e268a24f4ca0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd444f3769b4452a20f74b5341c1ac2" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34751 } } peers { permanent_uuid: "8e6be863daab4c0da5d4e268a24f4ca0" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46119 } } peers { permanent_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 45771 } }
I20260809 20:20:39.924170 22301 heartbeater.cc:503] Master 127.5.61.62:46463 was elected leader, sending a full tablet report...
I20260809 20:20:39.924608 22317 ts_tablet_manager.cc:1428] T 8ed86ca2f5584c3eaf73778c56822fa5 P 8e6be863daab4c0da5d4e268a24f4ca0: Time spent starting tablet: real 0.014s user 0.007s sys 0.005s
I20260809 20:20:39.927603 22152 heartbeater.cc:503] Master 127.5.61.62:46463 was elected leader, sending a full tablet report...
I20260809 20:20:39.928344 22318 ts_tablet_manager.cc:1428] T 8ed86ca2f5584c3eaf73778c56822fa5 P 3bd444f3769b4452a20f74b5341c1ac2: Time spent starting tablet: real 0.016s user 0.010s sys 0.006s
I20260809 20:20:39.931319 22321 raft_consensus.cc:348] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd444f3769b4452a20f74b5341c1ac2" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34751 } } peers { permanent_uuid: "8e6be863daab4c0da5d4e268a24f4ca0" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46119 } } peers { permanent_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 45771 } }
I20260809 20:20:39.931849 22321 raft_consensus.cc:374] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:20:39.931967 22321 raft_consensus.cc:729] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16db83d271ac4c2fa0f13fae0e6339d7, State: Initialized, Role: FOLLOWER
I20260809 20:20:39.932646 22321 consensus_queue.cc:260] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd444f3769b4452a20f74b5341c1ac2" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34751 } } peers { permanent_uuid: "8e6be863daab4c0da5d4e268a24f4ca0" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46119 } } peers { permanent_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 45771 } }
I20260809 20:20:39.934233 22321 ts_tablet_manager.cc:1428] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7: Time spent starting tablet: real 0.015s user 0.012s sys 0.000s
I20260809 20:20:39.996042 22325 raft_consensus.cc:482] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:20:39.996397 22325 raft_consensus.cc:504] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd444f3769b4452a20f74b5341c1ac2" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34751 } } peers { permanent_uuid: "8e6be863daab4c0da5d4e268a24f4ca0" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46119 } } peers { permanent_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 45771 } }
I20260809 20:20:39.998343 22325 leader_election.cc:288] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bd444f3769b4452a20f74b5341c1ac2 (127.5.61.2:34751), 8e6be863daab4c0da5d4e268a24f4ca0 (127.5.61.3:46119)
I20260809 20:20:40.008203 22314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:39.998916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:59981 (local address 127.5.61.3:46119)
0809 20:20:39.999122 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:39.999128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:39.999950 (+ 822us) server_negotiation.cc:407] Connection header received
0809 20:20:40.000197 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:40.000203 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:40.000283 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:40.000385 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:20:40.001482 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:40.002625 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:40.005755 (+ 3130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:40.007020 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:40.007165 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:40.007258 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:40.007793 (+ 535us) server_negotiation.cc:299] Negotiation successful
0809 20:20:40.007938 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 20:20:40.008205 22313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:39.998933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:35429 (local address 127.5.61.2:34751)
0809 20:20:39.999122 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:39.999128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:39.999951 (+ 823us) server_negotiation.cc:407] Connection header received
0809 20:20:40.000197 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:40.000203 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:40.000283 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:40.000395 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:20:40.001620 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:40.002685 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:40.005757 (+ 3072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:40.006966 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:40.007156 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:40.007256 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:40.007793 (+ 537us) server_negotiation.cc:299] Negotiation successful
0809 20:20:40.007935 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 20:20:40.008538 22326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:39.998749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:34751 (local address 127.5.61.1:35429)
0809 20:20:39.999839 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:39.999856 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:40.000029 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:40.000541 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:40.000547 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:40.000599 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:20:40.001371 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:40.001385 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:40.002806 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:40.002813 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:40.005574 (+ 2761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:40.005590 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:40.007344 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:40.007351 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:40.007634 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:40.007661 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 20:20:40.007823 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 20:20:40.007979 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":988,"mutex_wait_us":343,"spinlock_wait_cycles":52992,"thread_start_us":620,"threads_started":1}
I20260809 20:20:40.008574 22327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:39.998636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:46119 (local address 127.5.61.1:59981)
0809 20:20:39.999804 (+ 1168us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:39.999828 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 20:20:39.999988 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:20:40.000541 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:20:40.000547 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:20:40.000599 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:20:40.001303 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:40.001322 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:40.002778 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:40.002784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:40.005574 (+ 2790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:20:40.005590 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:20:40.007312 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:20:40.007324 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:20:40.007514 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:40.007637 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 20:20:40.007799 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 20:20:40.007979 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1003,"mutex_wait_us":244,"spinlock_wait_cycles":13440,"thread_start_us":729,"threads_started":1}
I20260809 20:20:40.009730 22105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed86ca2f5584c3eaf73778c56822fa5" candidate_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd444f3769b4452a20f74b5341c1ac2" is_pre_election: true
I20260809 20:20:40.009783 22248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed86ca2f5584c3eaf73778c56822fa5" candidate_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e6be863daab4c0da5d4e268a24f4ca0" is_pre_election: true
I20260809 20:20:40.010316 22248 raft_consensus.cc:2454] T 8ed86ca2f5584c3eaf73778c56822fa5 P 8e6be863daab4c0da5d4e268a24f4ca0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16db83d271ac4c2fa0f13fae0e6339d7 in term 0.
I20260809 20:20:40.010316 22105 raft_consensus.cc:2454] T 8ed86ca2f5584c3eaf73778c56822fa5 P 3bd444f3769b4452a20f74b5341c1ac2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16db83d271ac4c2fa0f13fae0e6339d7 in term 0.
I20260809 20:20:40.011109 21884 leader_election.cc:302] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 16db83d271ac4c2fa0f13fae0e6339d7, 3bd444f3769b4452a20f74b5341c1ac2, 8e6be863daab4c0da5d4e268a24f4ca0; no voters:
I20260809 20:20:40.011539 22325 raft_consensus.cc:2781] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:20:40.011780 22325 raft_consensus.cc:482] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:20:40.011896 22325 raft_consensus.cc:3037] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:20:40.014146 22325 raft_consensus.cc:504] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd444f3769b4452a20f74b5341c1ac2" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34751 } } peers { permanent_uuid: "8e6be863daab4c0da5d4e268a24f4ca0" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46119 } } peers { permanent_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 45771 } }
I20260809 20:20:40.014704 22325 leader_election.cc:288] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7 [CANDIDATE]: Term 1 election: Requested vote from peers 3bd444f3769b4452a20f74b5341c1ac2 (127.5.61.2:34751), 8e6be863daab4c0da5d4e268a24f4ca0 (127.5.61.3:46119)
I20260809 20:20:40.015125 22248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed86ca2f5584c3eaf73778c56822fa5" candidate_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e6be863daab4c0da5d4e268a24f4ca0"
I20260809 20:20:40.015182 22105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ed86ca2f5584c3eaf73778c56822fa5" candidate_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd444f3769b4452a20f74b5341c1ac2"
I20260809 20:20:40.015309 22248 raft_consensus.cc:3037] T 8ed86ca2f5584c3eaf73778c56822fa5 P 8e6be863daab4c0da5d4e268a24f4ca0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:20:40.015424 22105 raft_consensus.cc:3037] T 8ed86ca2f5584c3eaf73778c56822fa5 P 3bd444f3769b4452a20f74b5341c1ac2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:20:40.017612 22248 raft_consensus.cc:2454] T 8ed86ca2f5584c3eaf73778c56822fa5 P 8e6be863daab4c0da5d4e268a24f4ca0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16db83d271ac4c2fa0f13fae0e6339d7 in term 1.
I20260809 20:20:40.017848 22105 raft_consensus.cc:2454] T 8ed86ca2f5584c3eaf73778c56822fa5 P 3bd444f3769b4452a20f74b5341c1ac2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16db83d271ac4c2fa0f13fae0e6339d7 in term 1.
I20260809 20:20:40.018024 21884 leader_election.cc:302] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16db83d271ac4c2fa0f13fae0e6339d7, 8e6be863daab4c0da5d4e268a24f4ca0; no voters:
I20260809 20:20:40.018317 22325 raft_consensus.cc:2781] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:20:40.018767 22325 raft_consensus.cc:686] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7 [term 1 LEADER]: Becoming Leader. State: Replica: 16db83d271ac4c2fa0f13fae0e6339d7, State: Running, Role: LEADER
I20260809 20:20:40.019327 22325 consensus_queue.cc:237] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd444f3769b4452a20f74b5341c1ac2" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34751 } } peers { permanent_uuid: "8e6be863daab4c0da5d4e268a24f4ca0" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46119 } } peers { permanent_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 45771 } }
I20260809 20:20:40.023866 21795 catalog_manager.cc:5495] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16db83d271ac4c2fa0f13fae0e6339d7 (127.5.61.1). New cstate: current_term: 1 leader_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd444f3769b4452a20f74b5341c1ac2" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e6be863daab4c0da5d4e268a24f4ca0" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16db83d271ac4c2fa0f13fae0e6339d7" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 45771 } health_report { overall_health: HEALTHY } } }
I20260809 20:20:40.043784 22316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:40.038626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53426 (local address 127.5.61.1:45771)
0809 20:20:40.038821 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:40.038825 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:40.038842 (+ 17us) server_negotiation.cc:407] Connection header received
0809 20:20:40.038949 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:40.038952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:40.039024 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:40.039137 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:20:40.039541 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:40.040317 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:40.041783 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:40.042169 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:40.042262 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:40.042687 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:20:40.043207 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:20:40.043318 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:40.043397 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 20:20:40.043457 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 20:20:40.051386 22248 raft_consensus.cc:1261] T 8ed86ca2f5584c3eaf73778c56822fa5 P 8e6be863daab4c0da5d4e268a24f4ca0 [term 1 FOLLOWER]: Refusing update from remote peer 16db83d271ac4c2fa0f13fae0e6339d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:20:40.051404 22105 raft_consensus.cc:1261] T 8ed86ca2f5584c3eaf73778c56822fa5 P 3bd444f3769b4452a20f74b5341c1ac2 [term 1 FOLLOWER]: Refusing update from remote peer 16db83d271ac4c2fa0f13fae0e6339d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 20:20:40.052659 22302 tablet.cc:2369] T 8ed86ca2f5584c3eaf73778c56822fa5 P 8e6be863daab4c0da5d4e268a24f4ca0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:20:40.052531 22328 consensus_queue.cc:1035] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd444f3769b4452a20f74b5341c1ac2" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:20:40.052887 22325 consensus_queue.cc:1035] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e6be863daab4c0da5d4e268a24f4ca0" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 46119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:20:40.064776 22333 mvcc.cc:205] Tried to move back new op lower bound from 7316712816841281536 to 7316712816723623936. Current Snapshot: MvccSnapshot[applied={T|T < 7316712816841281536}]
I20260809 20:20:40.068548 22331 mvcc.cc:205] Tried to move back new op lower bound from 7316712816841281536 to 7316712816723623936. Current Snapshot: MvccSnapshot[applied={T|T < 7316712816841281536 or (T in {7316712816841281536})}]
I20260809 20:20:40.078955 21795 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:59774:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:20:40.079252 21795 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:59774:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:20:40.082935 21795 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477: Sending DeleteTablet for 3 replicas of tablet 8ed86ca2f5584c3eaf73778c56822fa5
I20260809 20:20:40.084132 22081 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ed86ca2f5584c3eaf73778c56822fa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:20:40 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:49640
I20260809 20:20:40.084285 22224 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ed86ca2f5584c3eaf73778c56822fa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:20:40 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:38082
I20260809 20:20:40.084323 21935 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ed86ca2f5584c3eaf73778c56822fa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:20:40 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:53416
I20260809 20:20:40.085063 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 21841
I20260809 20:20:40.086819 22342 tablet_replica.cc:330] T 8ed86ca2f5584c3eaf73778c56822fa5 P 3bd444f3769b4452a20f74b5341c1ac2: stopping tablet replica
I20260809 20:20:40.087807 22342 raft_consensus.cc:2229] T 8ed86ca2f5584c3eaf73778c56822fa5 P 3bd444f3769b4452a20f74b5341c1ac2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:20:40.088332 22342 raft_consensus.cc:2258] T 8ed86ca2f5584c3eaf73778c56822fa5 P 3bd444f3769b4452a20f74b5341c1ac2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:20:40.089108 22344 tablet_replica.cc:330] T 8ed86ca2f5584c3eaf73778c56822fa5 P 8e6be863daab4c0da5d4e268a24f4ca0: stopping tablet replica
I20260809 20:20:40.089514 22344 raft_consensus.cc:2229] T 8ed86ca2f5584c3eaf73778c56822fa5 P 8e6be863daab4c0da5d4e268a24f4ca0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:20:40.090035 22344 raft_consensus.cc:2258] T 8ed86ca2f5584c3eaf73778c56822fa5 P 8e6be863daab4c0da5d4e268a24f4ca0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:20:40.090476 22343 tablet_replica.cc:330] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7: stopping tablet replica
I20260809 20:20:40.090642 22342 ts_tablet_manager.cc:1905] T 8ed86ca2f5584c3eaf73778c56822fa5 P 3bd444f3769b4452a20f74b5341c1ac2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:20:40.090871 22343 raft_consensus.cc:2229] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:20:40.091585 22343 raft_consensus.cc:2258] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:20:40.093528 22344 ts_tablet_manager.cc:1905] T 8ed86ca2f5584c3eaf73778c56822fa5 P 8e6be863daab4c0da5d4e268a24f4ca0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:20:40.096921 22343 ts_tablet_manager.cc:1905] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:20:40.097122 22342 ts_tablet_manager.cc:1918] T 8ed86ca2f5584c3eaf73778c56822fa5 P 3bd444f3769b4452a20f74b5341c1ac2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:20:40.097283 22342 log.cc:1196] T 8ed86ca2f5584c3eaf73778c56822fa5 P 3bd444f3769b4452a20f74b5341c1ac2: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/8ed86ca2f5584c3eaf73778c56822fa5
I20260809 20:20:40.097856 22342 ts_tablet_manager.cc:1939] T 8ed86ca2f5584c3eaf73778c56822fa5 P 3bd444f3769b4452a20f74b5341c1ac2: Deleting consensus metadata
I20260809 20:20:40.099709 21783 catalog_manager.cc:4841] TS 3bd444f3769b4452a20f74b5341c1ac2 (127.5.61.2:34751): tablet 8ed86ca2f5584c3eaf73778c56822fa5 (table test-table [id=b37a29f863ae4887bdfc6496d5d09ae2]) successfully deleted
I20260809 20:20:40.105013 22343 ts_tablet_manager.cc:1918] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:20:40.105207 22343 log.cc:1196] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/8ed86ca2f5584c3eaf73778c56822fa5
I20260809 20:20:40.105625 22344 ts_tablet_manager.cc:1918] T 8ed86ca2f5584c3eaf73778c56822fa5 P 8e6be863daab4c0da5d4e268a24f4ca0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:20:40.105832 22343 ts_tablet_manager.cc:1939] T 8ed86ca2f5584c3eaf73778c56822fa5 P 16db83d271ac4c2fa0f13fae0e6339d7: Deleting consensus metadata
I20260809 20:20:40.105829 22344 log.cc:1196] T 8ed86ca2f5584c3eaf73778c56822fa5 P 8e6be863daab4c0da5d4e268a24f4ca0: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/8ed86ca2f5584c3eaf73778c56822fa5
I20260809 20:20:40.106442 22344 ts_tablet_manager.cc:1939] T 8ed86ca2f5584c3eaf73778c56822fa5 P 8e6be863daab4c0da5d4e268a24f4ca0: Deleting consensus metadata
I20260809 20:20:40.107646 21783 catalog_manager.cc:4841] TS 16db83d271ac4c2fa0f13fae0e6339d7 (127.5.61.1:45771): tablet 8ed86ca2f5584c3eaf73778c56822fa5 (table test-table [id=b37a29f863ae4887bdfc6496d5d09ae2]) successfully deleted
I20260809 20:20:40.108326 21780 catalog_manager.cc:4841] TS 8e6be863daab4c0da5d4e268a24f4ca0 (127.5.61.3:46119): tablet 8ed86ca2f5584c3eaf73778c56822fa5 (table test-table [id=b37a29f863ae4887bdfc6496d5d09ae2]) successfully deleted
I20260809 20:20:40.110267 22316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:40.086095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53434 (local address 127.5.61.1:45771)
0809 20:20:40.086339 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:40.086349 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:40.086644 (+ 295us) server_negotiation.cc:407] Connection header received
0809 20:20:40.086719 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:40.086726 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:40.086838 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:40.086960 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:40.089592 (+ 2632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:40.090857 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:40.093086 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:40.093757 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:40.093884 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:40.101952 (+ 8068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:40.101972 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:40.101992 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:40.102054 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:40.106545 (+ 4491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:40.107526 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:40.107535 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:40.107538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:40.107771 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:40.108615 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:40.108618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:40.108620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:40.109001 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:40.109305 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:40.109653 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 20:20:40.109899 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 20:20:40.110898 22003 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:20:40.207224 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 21841
I20260809 20:20:40.229068 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 22013
I20260809 20:20:40.243309 22313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:40.229939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49652 (local address 127.5.61.2:34751)
0809 20:20:40.230140 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:40.230146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:40.230166 (+ 20us) server_negotiation.cc:407] Connection header received
0809 20:20:40.230376 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:40.230381 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:40.230471 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:40.230579 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:40.232037 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:40.232762 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:40.234046 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:40.234416 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:40.234549 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:40.237430 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:40.237446 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:40.237466 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:40.237510 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:40.240429 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:40.241133 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:40.241140 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:40.241142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:40.241270 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:40.241752 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:40.241756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:40.241758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:40.242230 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:40.242473 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:40.242836 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 20:20:40.243028 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 20:20:40.244127 22147 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:20:40.353994 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 22013
I20260809 20:20:40.376093 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 22157
I20260809 20:20:40.392549 22314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:40.376959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38094 (local address 127.5.61.3:46119)
0809 20:20:40.377177 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:40.377181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:40.377333 (+ 152us) server_negotiation.cc:407] Connection header received
0809 20:20:40.377532 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:40.377536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:40.377601 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:40.377685 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:40.380627 (+ 2942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:40.381357 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:40.383236 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:40.383734 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:40.383833 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:40.386972 (+ 3139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:40.386994 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:40.387016 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:40.387075 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:40.390000 (+ 2925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:40.390729 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:40.390736 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:40.390738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:40.390863 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:40.391181 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:40.391184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:40.391186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:40.391513 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:40.391848 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:40.392006 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 20:20:40.392204 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 20:20:40.393150 22296 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:20:40.480183 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 22157
I20260809 20:20:40.501915 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 21756
I20260809 20:20:40.503043 21832 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:20:40.614177 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 21756
I20260809 20:20:40.639165 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:46463
--webserver_interface=127.5.61.62
--webserver_port=35987
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:32969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:46463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:20:40.966131 22349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:40.966463 22349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:40.966612 22349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:20:40.973768 22349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:20:40.973912 22349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:40.974026 22349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:40.974128 22349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:20:40.974211 22349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:20:40.983907 22349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:32969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:46463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:46463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=35987
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:40.985253 22349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:40.986611 22349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:40.998582 22354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:40.998565 22358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:40.998593 22355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:40.999444 22349 server_base.cc:1034] running on GCE node
I20260809 20:20:41.000263 22349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:41.001611 22349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:41.002857 22349 hybrid_clock.cc:648] HybridClock initialized: now 1786306841002819 us; error 63 us; skew 500 ppm
I20260809 20:20:41.003186 22349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:20:41 dist-test-slave-6k22 krb5kdc[21155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306841, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:41.006335 22349 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:20:41.007956 22349 webserver.cc:460] Webserver started at http://127.5.61.62:35987/ using document root <none> and password file <none>
I20260809 20:20:41.008728 22349 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:41.008881 22349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:20:41.014477 22349 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 20:20:41.017410 22366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:41.018792 22349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 20:20:41.018980 22349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "8c69b8e666d245ada106cc1db1ac8477"
format_stamp: "Formatted at 2026-08-09 20:20:35 on dist-test-slave-6k22"
server_key: "2e07a2de0c7414fd9eb5f6683382fc64"
server_key_iv: "c1133fc7cd642091af6bf4a9ec263ce4"
server_key_version: "encryptionkey@0"
I20260809 20:20:41.020020 22349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:41.034614 22349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:41.038847 22349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:41.039126 22349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:41.057381 22349 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:46463
I20260809 20:20:41.057426 22419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:46463 every 8 connection(s)
I20260809 20:20:41.058882 22349 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:20:41.065413 22420 sys_catalog.cc:263] Verifying existing consensus state
I20260809 20:20:41.065480 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 22349
I20260809 20:20:41.066318 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:45771
--local_ip_for_outbound_sockets=127.5.61.1
--tserver_master_addrs=127.5.61.62:46463
--webserver_port=41785
--webserver_interface=127.5.61.1
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:32969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 20:20:41.068740 22420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477: Bootstrap starting.
I20260809 20:20:41.092904 22420 log.cc:824] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477: Log is configured to *not* fsync() on all Append() calls
I20260809 20:20:41.109903 22420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 20:20:41.110440 22420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477: Bootstrap complete.
I20260809 20:20:41.119751 22420 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c69b8e666d245ada106cc1db1ac8477" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46463 } }
I20260809 20:20:41.120615 22420 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c69b8e666d245ada106cc1db1ac8477, State: Initialized, Role: FOLLOWER
I20260809 20:20:41.121150 22420 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c69b8e666d245ada106cc1db1ac8477" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46463 } }
I20260809 20:20:41.121384 22420 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:20:41.121515 22420 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:20:41.121701 22420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 2 FOLLOWER]: Advancing to term 3
I20260809 20:20:41.125747 22420 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c69b8e666d245ada106cc1db1ac8477" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46463 } }
I20260809 20:20:41.126279 22420 leader_election.cc:302] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c69b8e666d245ada106cc1db1ac8477; no voters:
I20260809 20:20:41.126930 22420 leader_election.cc:288] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 20:20:41.127197 22424 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 20:20:41.128340 22424 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [term 3 LEADER]: Becoming Leader. State: Replica: 8c69b8e666d245ada106cc1db1ac8477, State: Running, Role: LEADER
I20260809 20:20:41.129007 22420 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:20:41.129069 22424 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c69b8e666d245ada106cc1db1ac8477" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46463 } }
I20260809 20:20:41.135429 22425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8c69b8e666d245ada106cc1db1ac8477" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c69b8e666d245ada106cc1db1ac8477" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46463 } } }
I20260809 20:20:41.136204 22425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [sys.catalog]: This master's current role is: LEADER
I20260809 20:20:41.135902 22426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c69b8e666d245ada106cc1db1ac8477. Latest consensus state: current_term: 3 leader_uuid: "8c69b8e666d245ada106cc1db1ac8477" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c69b8e666d245ada106cc1db1ac8477" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 46463 } } }
I20260809 20:20:41.136639 22426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c69b8e666d245ada106cc1db1ac8477 [sys.catalog]: This master's current role is: LEADER
I20260809 20:20:41.138486 22438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:20:41.150403 22438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:20:41.151885 22438 catalog_manager.cc:1259] Loaded cluster ID: 84b3bfbcdc594f2db1e6498ba6dfebc9
I20260809 20:20:41.151978 22438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 20:20:41.153993 22438 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ffb08f529ad google::LogMessage::Fail() at ??:0
@ 0x7ffb08f5588a google::LogMessage::SendToLog() at ??:0
@ 0x7ffb08f524de google::LogMessage::Flush() at ??:0
@ 0x7ffb08f53a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ffb1e989512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7ffb1e986976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ffb1ea0e610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7ffb1ea0e439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7ffb0bb9e345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ffb0bba191c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7ffb0bba1799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7ffb0bb7ceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7ffb0ee566db start_thread at ??:0
@ 0x7ffb07dec61f clone at ??:0
W20260809 20:20:41.457595 22422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:41.457917 22422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:41.458087 22422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:20:41.465322 22422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:41.465474 22422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:41.465646 22422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:20:41.475361 22422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:32969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:45771
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=41785
--tserver_master_addrs=127.5.61.62:46463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:41.476673 22422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:41.478089 22422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:41.493328 22467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:41.493302 22471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:41.493275 22468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:41.493772 22422 server_base.cc:1034] running on GCE node
I20260809 20:20:41.494566 22422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:41.495512 22422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:41.496760 22422 hybrid_clock.cc:648] HybridClock initialized: now 1786306841496720 us; error 35 us; skew 500 ppm
I20260809 20:20:41.497068 22422 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:20:41 dist-test-slave-6k22 krb5kdc[21155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306841, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:41.500206 22422 init.cc:377] Logged in from keytab as kudu/127.5.61.1@KRBTEST.COM (short username kudu)
I20260809 20:20:41.501627 22422 webserver.cc:460] Webserver started at http://127.5.61.1:41785/ using document root <none> and password file <none>
I20260809 20:20:41.502331 22422 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:41.502475 22422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:20:41.507858 22422 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 20:20:41.510658 22479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:41.512089 22422 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:20:41.512281 22422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "16db83d271ac4c2fa0f13fae0e6339d7"
format_stamp: "Formatted at 2026-08-09 20:20:35 on dist-test-slave-6k22"
server_key: "410044219cbfc5ecfd1e5611a15035d6"
server_key_iv: "fb6ffcf585f70f4358959337f2bcbeaa"
server_key_version: "encryptionkey@0"
I20260809 20:20:41.513315 22422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:41.529049 22422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:41.532871 22422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:41.533185 22422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:41.534169 22422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:20:41.535544 22422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:20:41.535684 22422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:41.535836 22422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:20:41.535933 22422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:41.570458 22422 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:45771
I20260809 20:20:41.570547 22601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:45771 every 8 connection(s)
I20260809 20:20:41.572041 22422 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:20:41.576800 22604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:41.574940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.1:47437)
0809 20:20:41.575741 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:41.575948 (+ 207us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":296,"threads_started":1}
W20260809 20:20:41.577479 22602 heartbeater.cc:644] Failed to heartbeat to 127.5.61.62:46463 (0 consecutive failures): Network error: Failed to ping master at 127.5.61.62:46463: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
I20260809 20:20:41.578253 22604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:41.577960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.1:48281)
0809 20:20:41.578088 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:41.578154 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 20:20:41.579147 22604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:41.578757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.1:34435)
0809 20:20:41.578892 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:41.578988 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 20:20:41.579448 22602 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 20:20:41.579685 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 22422
I20260809 20:20:41.580374 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:34751
--local_ip_for_outbound_sockets=127.5.61.2
--tserver_master_addrs=127.5.61.62:46463
--webserver_port=40065
--webserver_interface=127.5.61.2
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:32969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:20:41.905741 22605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:41.905987 22605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:41.906121 22605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:20:41.913038 22605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:41.913190 22605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:41.913357 22605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:20:41.922696 22605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:32969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:34751
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=40065
--tserver_master_addrs=127.5.61.62:46463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:41.923933 22605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:41.925189 22605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:41.938046 22611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:41.938184 22610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:41.938126 22613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:41.939013 22605 server_base.cc:1034] running on GCE node
I20260809 20:20:41.939626 22605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:41.940510 22605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:41.941732 22605 hybrid_clock.cc:648] HybridClock initialized: now 1786306841941683 us; error 62 us; skew 500 ppm
I20260809 20:20:41.942037 22605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:20:41 dist-test-slave-6k22 krb5kdc[21155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306841, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:41.945091 22605 init.cc:377] Logged in from keytab as kudu/127.5.61.2@KRBTEST.COM (short username kudu)
I20260809 20:20:41.946362 22605 webserver.cc:460] Webserver started at http://127.5.61.2:40065/ using document root <none> and password file <none>
I20260809 20:20:41.947045 22605 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:41.947150 22605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:20:41.952447 22605 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:20:41.954970 22622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:41.956197 22605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 20:20:41.956312 22605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "3bd444f3769b4452a20f74b5341c1ac2"
format_stamp: "Formatted at 2026-08-09 20:20:36 on dist-test-slave-6k22"
server_key: "53ad4d1cac2ccc6db4b91968915fb390"
server_key_iv: "e252fc5413065112e812f8de2bc48976"
server_key_version: "encryptionkey@0"
I20260809 20:20:41.957209 22605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:41.977643 22605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:41.981616 22605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:41.981848 22605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:41.982766 22605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:20:41.984028 22605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:20:41.984113 22605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:41.984201 22605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:20:41.984256 22605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:42.017267 22605 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:34751
I20260809 20:20:42.017354 22745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:34751 every 8 connection(s)
I20260809 20:20:42.018834 22605 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:20:42.023808 22748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:42.021908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.2:55809)
0809 20:20:42.022620 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:42.022848 (+ 228us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":266,"threads_started":1}
W20260809 20:20:42.024472 22746 heartbeater.cc:644] Failed to heartbeat to 127.5.61.62:46463 (0 consecutive failures): Network error: Failed to ping master at 127.5.61.62:46463: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
I20260809 20:20:42.025224 22748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:42.024936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.2:57709)
0809 20:20:42.025063 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:42.025126 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 20:20:42.026024 22748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:42.025729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.2:53995)
0809 20:20:42.025853 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:42.025921 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 20:20:42.026279 22746 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 20:20:42.028255 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 22605
I20260809 20:20:42.028858 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:46119
--local_ip_for_outbound_sockets=127.5.61.3
--tserver_master_addrs=127.5.61.62:46463
--webserver_port=38121
--webserver_interface=127.5.61.3
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:32969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:20:42.357403 22749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:42.357661 22749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:42.357821 22749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:20:42.364984 22749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:42.365108 22749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:42.365290 22749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:20:42.374706 22749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:32969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:46119
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=38121
--tserver_master_addrs=127.5.61.62:46463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:42.376057 22749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:42.377384 22749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:42.390777 22756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:42.390715 22757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:42.390779 22759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:42.391506 22749 server_base.cc:1034] running on GCE node
I20260809 20:20:42.392119 22749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:42.393029 22749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:42.394225 22749 hybrid_clock.cc:648] HybridClock initialized: now 1786306842394179 us; error 41 us; skew 500 ppm
I20260809 20:20:42.394544 22749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:20:42 dist-test-slave-6k22 krb5kdc[21155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306842, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:42.397689 22749 init.cc:377] Logged in from keytab as kudu/127.5.61.3@KRBTEST.COM (short username kudu)
I20260809 20:20:42.399134 22749 webserver.cc:460] Webserver started at http://127.5.61.3:38121/ using document root <none> and password file <none>
I20260809 20:20:42.399886 22749 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:42.399987 22749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:20:42.405190 22749 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:20:42.407888 22767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:42.409235 22749 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:20:42.409408 22749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "8e6be863daab4c0da5d4e268a24f4ca0"
format_stamp: "Formatted at 2026-08-09 20:20:37 on dist-test-slave-6k22"
server_key: "4f5abad1086689b31f9c9c9beac7a159"
server_key_iv: "a9e34eea7de7dd903398d049a504b575"
server_key_version: "encryptionkey@0"
I20260809 20:20:42.410373 22749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:42.459050 22749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:42.463325 22749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:42.463635 22749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:42.464644 22749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:20:42.466037 22749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:20:42.466159 22749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:42.466287 22749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:20:42.466378 22749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:42.499661 22749 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:46119
I20260809 20:20:42.499722 22888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:46119 every 8 connection(s)
I20260809 20:20:42.501132 22749 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:20:42.505744 22892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:42.503978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.3:53953)
0809 20:20:42.504660 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:42.504879 (+ 219us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":264,"threads_started":1}
W20260809 20:20:42.506331 22889 heartbeater.cc:644] Failed to heartbeat to 127.5.61.62:46463 (0 consecutive failures): Network error: Failed to ping master at 127.5.61.62:46463: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
I20260809 20:20:42.506314 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 22749
I20260809 20:20:42.507277 22892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:42.506821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.3:43921)
0809 20:20:42.506984 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:42.507110 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 20:20:42.508138 22892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:42.507811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.3:40551)
0809 20:20:42.507950 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:42.508036 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 20:20:42.508508 22889 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 20:20:42.580737 22895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:42.579936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.1:57459)
0809 20:20:42.580422 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:42.580505 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":277,"threads_started":1}
I20260809 20:20:43.027542 22896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:43.026819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.2:55505)
0809 20:20:43.027221 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:43.027319 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":202,"threads_started":1}
I20260809 20:20:43.510030 22897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:43.509010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.3:41481)
0809 20:20:43.509586 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:43.509717 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":336,"threads_started":1}
I20260809 20:20:43.582324 22898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:43.581595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.1:57965)
0809 20:20:43.582067 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:43.582157 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":249,"threads_started":1}
I20260809 20:20:44.029119 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:44.028410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.2:45555)
0809 20:20:44.028852 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:44.028939 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":238,"threads_started":1}
I20260809 20:20:44.511705 22900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:44.510960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.3:59755)
0809 20:20:44.511399 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:44.511477 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":225,"threads_started":1}
I20260809 20:20:44.583854 22901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:44.583125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.1:54531)
0809 20:20:44.583550 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:44.583680 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":229,"threads_started":1}
I20260809 20:20:45.030623 22902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:45.029957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.2:35049)
0809 20:20:45.030370 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:45.030447 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":225,"threads_started":1}
I20260809 20:20:45.513288 22903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:45.512523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.3:50473)
0809 20:20:45.513013 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:45.513114 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":279,"threads_started":1}
I20260809 20:20:45.585597 22905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:45.584794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.1:43019)
0809 20:20:45.585319 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:45.585408 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":298,"threads_started":1}
I20260809 20:20:46.032377 22906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:46.031625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.2:35187)
0809 20:20:46.032096 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:46.032175 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":229,"threads_started":1}
I20260809 20:20:46.514961 22907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:46.514181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.3:34445)
0809 20:20:46.514705 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:46.514785 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":250,"threads_started":1}
I20260809 20:20:46.587253 22908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:46.586537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.1:38059)
0809 20:20:46.586985 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:46.587079 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":238,"threads_started":1}
I20260809 20:20:47.033995 22909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:47.033284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.2:45159)
0809 20:20:47.033753 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:47.033831 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":252,"threads_started":1}
I20260809 20:20:47.516623 22910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:47.515790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.3:33925)
0809 20:20:47.516360 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:47.516443 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":267,"threads_started":1}
I20260809 20:20:47.589021 22911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:47.588160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.1:58851)
0809 20:20:47.588756 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:47.588844 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":315,"threads_started":1}
I20260809 20:20:48.035473 22912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:48.034792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.2:33399)
0809 20:20:48.035205 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:48.035317 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":217,"threads_started":1}
I20260809 20:20:48.518296 22913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:48.517497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.3:49101)
0809 20:20:48.518033 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:48.518129 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":310,"threads_started":1}
I20260809 20:20:48.590840 22914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:48.590008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.1:36027)
0809 20:20:48.590530 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:48.590609 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":249,"threads_started":1}
I20260809 20:20:49.037194 22915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:49.036424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.2:57567)
0809 20:20:49.036899 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:49.036977 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":205,"threads_started":1}
I20260809 20:20:49.519994 22916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:49.519154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.3:59295)
0809 20:20:49.519677 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:49.519769 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":229,"threads_started":1}
I20260809 20:20:49.592525 22917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:49.591817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.1:33789)
0809 20:20:49.592268 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:49.592359 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":230,"threads_started":1}
I20260809 20:20:50.038759 22918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:50.038060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.2:44157)
0809 20:20:50.038514 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:50.038591 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":240,"threads_started":1}
I20260809 20:20:50.521553 22919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:50.520861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.3:36347)
0809 20:20:50.521290 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:50.521378 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":224,"threads_started":1}
I20260809 20:20:50.594360 22920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:50.593470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.1:40981)
0809 20:20:50.594078 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:50.594174 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":282,"threads_started":1}
I20260809 20:20:51.040361 22921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:51.039658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.2:40219)
0809 20:20:51.040112 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:51.040191 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":223,"threads_started":1}
I20260809 20:20:51.523146 22922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:51.522441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.3:48217)
0809 20:20:51.522883 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:51.522961 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":219,"threads_started":1}
I20260809 20:20:51.596107 22923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:51.595357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.1:45869)
0809 20:20:51.595866 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:51.595942 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":292,"threads_started":1}
I20260809 20:20:52.041903 22924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:52.041227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.2:34725)
0809 20:20:52.041656 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:52.041745 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":251,"threads_started":1}
I20260809 20:20:52.525003 22925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:52.524199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.3:43407)
0809 20:20:52.524748 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:52.524825 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":315,"threads_started":1}
I20260809 20:20:52.597719 22926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:52.597017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.1:49895)
0809 20:20:52.597453 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:52.597542 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":223,"threads_started":1}
I20260809 20:20:53.043707 22927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:53.042904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.2:54109)
0809 20:20:53.043418 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:53.043510 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":281,"threads_started":1}
I20260809 20:20:53.526722 22928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:53.525977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.3:37653)
0809 20:20:53.526419 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:53.526498 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":233,"threads_started":1}
I20260809 20:20:53.599377 22929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:53.598645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.1:58427)
0809 20:20:53.599116 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:53.599199 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":244,"threads_started":1}
I20260809 20:20:54.045454 22930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:54.044630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.2:55539)
0809 20:20:54.045215 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:54.045298 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":280,"threads_started":1}
I20260809 20:20:54.528478 22931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:54.527799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.3:54021)
0809 20:20:54.528228 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:54.528309 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":216,"threads_started":1}
I20260809 20:20:54.600989 22932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:54.600260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.1:58339)
0809 20:20:54.600709 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:54.600793 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":224,"threads_started":1}
I20260809 20:20:55.047385 22933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:55.046517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.2:54565)
0809 20:20:55.047106 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:55.047189 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":306,"threads_started":1}
I20260809 20:20:55.530177 22934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:55.529354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.3:47437)
0809 20:20:55.529904 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:55.529991 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":248,"threads_started":1}
I20260809 20:20:55.602696 22935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:55.601952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.1:47647)
0809 20:20:55.602404 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:55.602497 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":231,"threads_started":1}
I20260809 20:20:56.049088 22936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:56.048255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.2:45809)
0809 20:20:56.048804 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:56.048902 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":228,"threads_started":1}
I20260809 20:20:56.531751 22937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:56.531007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.3:50387)
0809 20:20:56.531431 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:56.531508 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":215,"threads_started":1}
I20260809 20:20:56.604588 22938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:56.603833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.1:33747)
0809 20:20:56.604293 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:56.604409 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":246,"threads_started":1}
I20260809 20:20:57.051141 22939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:57.050290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.2:38985)
0809 20:20:57.050783 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:57.050909 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":237,"threads_started":1}
I20260809 20:20:57.517138 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 22422
I20260809 20:20:57.533546 22942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:57.532787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:46463 (local address 127.5.61.3:33723)
0809 20:20:57.533294 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0809 20:20:57.533378 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":283,"threads_started":1}
I20260809 20:20:57.541627 22941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:57.518622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48600 (local address 127.5.61.1:45771)
0809 20:20:57.519393 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:57.519408 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:57.519451 (+ 43us) server_negotiation.cc:407] Connection header received
0809 20:20:57.521116 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:57.521126 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:57.521544 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:57.521896 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:57.523500 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:57.524755 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:57.525996 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:57.526665 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:57.526908 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:57.535206 (+ 8298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:57.535215 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:57.535231 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:57.535267 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:57.538130 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:57.538905 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:57.538912 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:57.538918 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:57.539085 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:57.539507 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:57.539513 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:57.539516 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:57.540108 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:57.540432 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:57.540915 (+ 483us) server_negotiation.cc:299] Negotiation successful
0809 20:20:57.541349 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":397,"threads_started":1}
I20260809 20:20:57.543059 22596 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:20:57.629101 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 22422
I20260809 20:20:57.650341 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 22605
I20260809 20:20:57.668762 22945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:57.651356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48690 (local address 127.5.61.2:34751)
0809 20:20:57.652017 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:57.652050 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:57.652096 (+ 46us) server_negotiation.cc:407] Connection header received
0809 20:20:57.653771 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:57.653780 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:57.654193 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:57.654494 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:57.656092 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:57.657295 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:57.658483 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:57.659127 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:57.659254 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:57.662030 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:57.662038 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:57.662054 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:57.662097 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:57.665373 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:57.665993 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:57.665999 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:57.666005 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:57.666165 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:57.666581 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:57.666584 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:57.666586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:57.667095 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:57.667545 (+ 450us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:57.668013 (+ 468us) server_negotiation.cc:299] Negotiation successful
0809 20:20:57.668456 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":283,"threads_started":1}
I20260809 20:20:57.670171 22741 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:20:57.755090 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 22605
I20260809 20:20:57.776654 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 22749
I20260809 20:20:57.794642 22947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:57.777684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51048 (local address 127.5.61.3:46119)
0809 20:20:57.778255 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:57.778288 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:57.778331 (+ 43us) server_negotiation.cc:407] Connection header received
0809 20:20:57.780516 (+ 2185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:57.780527 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:57.780987 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:57.781387 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:57.782716 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:57.783996 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:57.785291 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:57.785964 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:57.786099 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:57.788665 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:57.788672 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:57.788687 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:57.788723 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:57.791810 (+ 3087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:57.792391 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:57.792397 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:57.792402 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:57.792576 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:57.792892 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:57.792895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:57.792897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:57.793411 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:57.793632 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:57.793933 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 20:20:57.794341 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":253,"threads_started":1}
I20260809 20:20:57.796068 22884 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:20:57.880190 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 22749
2026-08-09T20:20:57Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24343 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:20:57 dist-test-slave-6k22 krb5kdc[22955](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:20:57 dist-test-slave-6k22 krb5kdc[22955](info): set up 2 sockets
Aug 09 20:20:57 dist-test-slave-6k22 krb5kdc[22955](info): commencing operation
krb5kdc: starting...
W20260809 20:20:59.167820 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:20:59 dist-test-slave-6k22 krb5kdc[22955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:20:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:20:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:20:59.447790 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:41859
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:41859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:20:59.788218 22970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:20:59.788522 22970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:20:59.788699 22970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:20:59.795701 22970 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 20:20:59.795794 22970 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 20:20:59.795874 22970 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 20:20:59.795969 22970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:20:59.796058 22970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:20:59.796175 22970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:20:59.796247 22970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:20:59.796330 22970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:20:59.805998 22970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:41859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:41859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:20:59.807281 22970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:20:59.808643 22970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:20:59.820670 22975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:59.820699 22979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:20:59.820775 22976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:20:59.821480 22970 server_base.cc:1034] running on GCE node
I20260809 20:20:59.822281 22970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:20:59.823666 22970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:20:59.824910 22970 hybrid_clock.cc:648] HybridClock initialized: now 1786306859824836 us; error 43 us; skew 500 ppm
I20260809 20:20:59.825217 22970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:20:59 dist-test-slave-6k22 krb5kdc[22955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306859, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:20:59.828446 22970 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:20:59.829926 22970 webserver.cc:460] Webserver started at http://127.5.61.62:32847/ using document root <none> and password file <none>
I20260809 20:20:59.830585 22970 fs_manager.cc:359] Metadata directory not provided
I20260809 20:20:59.830672 22970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:20:59.830942 22970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:20:59.833956 22970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "8ef1eae90ca245239d5944f2d0e0d906"
format_stamp: "Formatted at 2026-08-09 20:20:59 on dist-test-slave-6k22"
server_key: "cb7a94fcb2a7d1a27eb7c976fca49b0e"
server_key_iv: "b069ff8c22cc92a1c33d495ff4cca014"
server_key_version: "encryptionkey@0"
I20260809 20:20:59.834688 22970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "8ef1eae90ca245239d5944f2d0e0d906"
format_stamp: "Formatted at 2026-08-09 20:20:59 on dist-test-slave-6k22"
server_key: "cb7a94fcb2a7d1a27eb7c976fca49b0e"
server_key_iv: "b069ff8c22cc92a1c33d495ff4cca014"
server_key_version: "encryptionkey@0"
I20260809 20:20:59.839640 22970 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 20:20:59.843022 22987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:20:59.844385 22970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:20:59.844547 22970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "8ef1eae90ca245239d5944f2d0e0d906"
format_stamp: "Formatted at 2026-08-09 20:20:59 on dist-test-slave-6k22"
server_key: "cb7a94fcb2a7d1a27eb7c976fca49b0e"
server_key_iv: "b069ff8c22cc92a1c33d495ff4cca014"
server_key_version: "encryptionkey@0"
I20260809 20:20:59.844743 22970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:20:59.860802 22970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:20:59.864609 22970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:20:59.864882 22970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:20:59.883258 22970 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:41859
I20260809 20:20:59.883277 23048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:41859 every 8 connection(s)
I20260809 20:20:59.884585 22970 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:20:59.888266 23049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:20:59.894665 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 22970
I20260809 20:20:59.894905 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:20:59.895586 5364 external_mini_cluster.cc:1428] Setting key e150bed6988dfb88549de35cd68eb124
I20260809 20:20:59.900544 23049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906: Bootstrap starting.
I20260809 20:20:59.904040 23049 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906: Neither blocks nor log segments found. Creating new log.
I20260809 20:20:59.905181 23049 log.cc:824] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906: Log is configured to *not* fsync() on all Append() calls
I20260809 20:20:59.908978 23049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906: No bootstrap required, opened a new log
Aug 09 20:20:59 dist-test-slave-6k22 krb5kdc[22955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:20:59.918061 23049 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef1eae90ca245239d5944f2d0e0d906" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41859 } }
I20260809 20:20:59.918555 23049 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:20:59.918701 23049 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef1eae90ca245239d5944f2d0e0d906, State: Initialized, Role: FOLLOWER
I20260809 20:20:59.919628 23053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:20:59.897768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49848 (local address 127.5.61.62:41859)
0809 20:20:59.898588 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0809 20:20:59.898599 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:20:59.898636 (+ 37us) server_negotiation.cc:407] Connection header received
0809 20:20:59.900355 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:20:59.900384 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:20:59.900789 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:20:59.901157 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:20:59.902665 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:59.904000 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:59.905392 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:20:59.906468 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:20:59.906631 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:20:59.912381 (+ 5750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:20:59.912389 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:20:59.912402 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:20:59.912455 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:20:59.915475 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:59.916192 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:59.916201 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:59.916207 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:59.916380 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:20:59.916824 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:20:59.916830 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:20:59.916833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:20:59.917338 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:20:59.917636 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0809 20:20:59.918082 (+ 446us) server_negotiation.cc:299] Negotiation successful
0809 20:20:59.918568 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":378,"threads_started":1}
I20260809 20:20:59.919430 23049 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef1eae90ca245239d5944f2d0e0d906" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41859 } }
I20260809 20:20:59.919839 23049 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:20:59.920053 23049 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:20:59.920298 23049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:20:59.923146 23049 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef1eae90ca245239d5944f2d0e0d906" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41859 } }
I20260809 20:20:59.923748 23049 leader_election.cc:302] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ef1eae90ca245239d5944f2d0e0d906; no voters:
I20260809 20:20:59.924393 23049 leader_election.cc:288] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:20:59.924494 23055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:20:59.925290 23055 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 1 LEADER]: Becoming Leader. State: Replica: 8ef1eae90ca245239d5944f2d0e0d906, State: Running, Role: LEADER
I20260809 20:20:59.925788 23055 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef1eae90ca245239d5944f2d0e0d906" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41859 } }
I20260809 20:20:59.926681 23049 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:20:59.928804 23056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ef1eae90ca245239d5944f2d0e0d906" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef1eae90ca245239d5944f2d0e0d906" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41859 } } }
I20260809 20:20:59.929360 23056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [sys.catalog]: This master's current role is: LEADER
I20260809 20:20:59.930297 23059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ef1eae90ca245239d5944f2d0e0d906. Latest consensus state: current_term: 1 leader_uuid: "8ef1eae90ca245239d5944f2d0e0d906" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef1eae90ca245239d5944f2d0e0d906" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41859 } } }
I20260809 20:20:59.930724 23059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [sys.catalog]: This master's current role is: LEADER
I20260809 20:20:59.932991 23062 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:20:59.939495 23062 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:20:59.948388 23062 catalog_manager.cc:1347] Generated new cluster ID: 67627d0ed1f84966b661aed141f857a7
I20260809 20:20:59.948515 23062 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:21:00.002718 23062 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:21:00.004725 23062 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:21:00.011752 23062 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906: Generated new TSK 0
I20260809 20:21:00.012740 23062 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.61.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
I20260809 20:21:00.082780 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:41859
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:21:00.449193 23082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:21:00.449502 23082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:21:00.449625 23082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:21:00.456723 23082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:21:00.456821 23082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:21:00.456928 23082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:21:00.466403 23082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:41859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:21:00.467657 23082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:21:00.468916 23082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:21:00.482048 23091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:21:00.482024 23089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:21:00.482283 23088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:21:00.482807 23082 server_base.cc:1034] running on GCE node
I20260809 20:21:00.483438 23082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:21:00.484378 23082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:21:00.485607 23082 hybrid_clock.cc:648] HybridClock initialized: now 1786306860485543 us; error 50 us; skew 500 ppm
I20260809 20:21:00.485987 23082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:21:00 dist-test-slave-6k22 krb5kdc[22955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306860, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:21:00.489403 23082 init.cc:377] Logged in from keytab as kudu/127.5.61.1@KRBTEST.COM (short username kudu)
I20260809 20:21:00.490841 23082 webserver.cc:460] Webserver started at http://127.5.61.1:40405/ using document root <none> and password file <none>
I20260809 20:21:00.491537 23082 fs_manager.cc:359] Metadata directory not provided
I20260809 20:21:00.491741 23082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:21:00.492082 23082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:21:00.495131 23082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "8ebf53713a3143e581f3e40def807bb5"
format_stamp: "Formatted at 2026-08-09 20:21:00 on dist-test-slave-6k22"
server_key: "5f4f6ce7ff00d0d10209fa47881a28f6"
server_key_iv: "b2ffe81e908d47d898b5f33299bd776f"
server_key_version: "encryptionkey@0"
I20260809 20:21:00.496012 23082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "8ebf53713a3143e581f3e40def807bb5"
format_stamp: "Formatted at 2026-08-09 20:21:00 on dist-test-slave-6k22"
server_key: "5f4f6ce7ff00d0d10209fa47881a28f6"
server_key_iv: "b2ffe81e908d47d898b5f33299bd776f"
server_key_version: "encryptionkey@0"
I20260809 20:21:00.501339 23082 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 20:21:00.504951 23099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:00.506294 23082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:21:00.506554 23082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "8ebf53713a3143e581f3e40def807bb5"
format_stamp: "Formatted at 2026-08-09 20:21:00 on dist-test-slave-6k22"
server_key: "5f4f6ce7ff00d0d10209fa47881a28f6"
server_key_iv: "b2ffe81e908d47d898b5f33299bd776f"
server_key_version: "encryptionkey@0"
I20260809 20:21:00.506752 23082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:21:00.520220 23082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:21:00.523926 23082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:21:00.524207 23082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:21:00.525084 23082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:21:00.526292 23082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:21:00.526363 23082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:00.526515 23082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:21:00.526584 23082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:00.559878 23082 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:44147
I20260809 20:21:00.559949 23241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:44147 every 8 connection(s)
I20260809 20:21:00.561153 23082 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:21:00.569438 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 23082
I20260809 20:21:00.569669 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:21:00.570366 5364 external_mini_cluster.cc:1428] Setting key 756546cdd52afafb2823d06da23002dc
Aug 09 20:21:00 dist-test-slave-6k22 krb5kdc[22955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306860, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:21:00.581305 23244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:00.563641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:60753 (local address 127.5.61.62:41859)
0809 20:21:00.564113 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:00.564118 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:00.564907 (+ 789us) server_negotiation.cc:407] Connection header received
0809 20:21:00.566717 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:00.566723 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:00.566829 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:00.566969 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:21:00.568610 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:00.569640 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:00.571379 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:00.571855 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:00.572010 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:00.575350 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:21:00.575355 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:21:00.575359 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:21:00.575393 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:21:00.578064 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:00.578890 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:00.578895 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:00.578897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:00.579022 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:00.579442 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:00.579449 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:00.579452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:00.579781 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:21:00.579967 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:00.580682 (+ 715us) server_negotiation.cc:299] Negotiation successful
0809 20:21:00.580937 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":260,"threads_started":1}
I20260809 20:21:00.582214 23245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:00.564022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.1:60753)
0809 20:21:00.564744 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:00.564790 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 20:21:00.566434 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:21:00.567225 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:21:00.567233 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:21:00.567857 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:21:00.568441 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:00.568458 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:00.569839 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:00.569843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:00.571226 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:00.571240 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:00.572095 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:00.572103 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:00.572350 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:00.573025 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:21:00.573047 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:21:00.575143 (+ 2096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:21:00.578230 (+ 3087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:21:00.578239 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:21:00.578255 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:21:00.578749 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:21:00.579131 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:21:00.579136 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:21:00.579139 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:21:00.579324 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:21:00.579940 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:21:00.579947 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:21:00.580299 (+ 352us) client_negotiation.cc:769] Sending connection context
0809 20:21:00.580576 (+ 277us) client_negotiation.cc:240] Negotiation successful
0809 20:21:00.581094 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":309,"threads_started":1}
I20260809 20:21:00.583755 23242 heartbeater.cc:347] Connected to a master server at 127.5.61.62:41859
I20260809 20:21:00.584089 23242 heartbeater.cc:464] Registering TS with master...
I20260809 20:21:00.584952 23242 heartbeater.cc:511] Master 127.5.61.62:41859 requested a full tablet report, sending...
I20260809 20:21:00.587031 23008 ts_manager.cc:194] Registered new tserver with Master: 8ebf53713a3143e581f3e40def807bb5 (127.5.61.1:44147)
I20260809 20:21:00.588516 23008 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.1@KRBTEST.COM'} at 127.5.61.1:60753
WARNING: no policy specified for kudu/127.5.61.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
I20260809 20:21:00.631124 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:41859
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:21:00.974251 23249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:21:00.974535 23249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:21:00.974694 23249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:21:00.982040 23249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:21:00.982167 23249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:21:00.982347 23249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:21:00.992168 23249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:41859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:21:00.993510 23249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:21:00.995069 23249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:21:01.008713 23258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:21:01.008810 23255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:21:01.008800 23256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:21:01.009786 23249 server_base.cc:1034] running on GCE node
I20260809 20:21:01.010483 23249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:21:01.011515 23249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:21:01.012776 23249 hybrid_clock.cc:648] HybridClock initialized: now 1786306861012722 us; error 44 us; skew 500 ppm
I20260809 20:21:01.013105 23249 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:21:01 dist-test-slave-6k22 krb5kdc[22955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306861, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:21:01.016244 23249 init.cc:377] Logged in from keytab as kudu/127.5.61.2@KRBTEST.COM (short username kudu)
I20260809 20:21:01.017807 23249 webserver.cc:460] Webserver started at http://127.5.61.2:42803/ using document root <none> and password file <none>
I20260809 20:21:01.018512 23249 fs_manager.cc:359] Metadata directory not provided
I20260809 20:21:01.018672 23249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:21:01.019021 23249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:21:01.022064 23249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "17954d03dc0144dfafdfa2d9383e8027"
format_stamp: "Formatted at 2026-08-09 20:21:01 on dist-test-slave-6k22"
server_key: "51d27605186ccefec7fe380c09ea5db5"
server_key_iv: "c28426eb1bab84ba6634008194d22c9d"
server_key_version: "encryptionkey@0"
I20260809 20:21:01.022868 23249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "17954d03dc0144dfafdfa2d9383e8027"
format_stamp: "Formatted at 2026-08-09 20:21:01 on dist-test-slave-6k22"
server_key: "51d27605186ccefec7fe380c09ea5db5"
server_key_iv: "c28426eb1bab84ba6634008194d22c9d"
server_key_version: "encryptionkey@0"
I20260809 20:21:01.028221 23249 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260809 20:21:01.031769 23266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:01.033095 23249 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 20:21:01.033301 23249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "17954d03dc0144dfafdfa2d9383e8027"
format_stamp: "Formatted at 2026-08-09 20:21:01 on dist-test-slave-6k22"
server_key: "51d27605186ccefec7fe380c09ea5db5"
server_key_iv: "c28426eb1bab84ba6634008194d22c9d"
server_key_version: "encryptionkey@0"
I20260809 20:21:01.033557 23249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:21:01.050383 23249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:21:01.054738 23249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:21:01.055032 23249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:21:01.055991 23249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:21:01.057260 23249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:21:01.057358 23249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:01.057515 23249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:21:01.057586 23249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:01.091845 23249 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:37737
I20260809 20:21:01.091936 23394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:37737 every 8 connection(s)
I20260809 20:21:01.093102 23249 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:21:01.096580 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 23249
I20260809 20:21:01.096803 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:21:01.097457 5364 external_mini_cluster.cc:1428] Setting key 7bf85c2f3246e4d4edd4122623c0779f
Aug 09 20:21:01 dist-test-slave-6k22 krb5kdc[22955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306861, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:21:01.114413 23397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:01.095657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:47425 (local address 127.5.61.62:41859)
0809 20:21:01.096168 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:01.096174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:01.097239 (+ 1065us) server_negotiation.cc:407] Connection header received
0809 20:21:01.099541 (+ 2302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:01.099546 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:01.099651 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:01.099773 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:21:01.101217 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.101962 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.103472 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.104022 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.104156 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:01.108080 (+ 3924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:21:01.108084 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:21:01.108087 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:21:01.108117 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:21:01.110956 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:01.111914 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:01.111922 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:01.111928 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:01.112093 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:01.112499 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:01.112507 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:01.112512 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:01.112883 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:21:01.113030 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:01.113670 (+ 640us) server_negotiation.cc:299] Negotiation successful
0809 20:21:01.113918 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":258,"threads_started":1}
I20260809 20:21:01.114880 23398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:01.096127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.2:47425)
0809 20:21:01.097057 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:01.097107 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 20:21:01.099258 (+ 2151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:21:01.100010 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:21:01.100018 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:21:01.100514 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:21:01.101055 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:01.101072 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.102126 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.102130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:01.103331 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:01.103342 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.104260 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.104267 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:01.104500 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:01.105169 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:21:01.105190 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:21:01.107860 (+ 2670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:21:01.111113 (+ 3253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:21:01.111121 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:21:01.111147 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:21:01.111803 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:21:01.112203 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:21:01.112206 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:21:01.112208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:21:01.112389 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:21:01.113005 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:21:01.113010 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:21:01.113294 (+ 284us) client_negotiation.cc:769] Sending connection context
0809 20:21:01.113520 (+ 226us) client_negotiation.cc:240] Negotiation successful
0809 20:21:01.113857 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":325,"threads_started":1}
I20260809 20:21:01.116425 23395 heartbeater.cc:347] Connected to a master server at 127.5.61.62:41859
I20260809 20:21:01.116744 23395 heartbeater.cc:464] Registering TS with master...
I20260809 20:21:01.117645 23395 heartbeater.cc:511] Master 127.5.61.62:41859 requested a full tablet report, sending...
I20260809 20:21:01.119254 23008 ts_manager.cc:194] Registered new tserver with Master: 17954d03dc0144dfafdfa2d9383e8027 (127.5.61.2:37737)
I20260809 20:21:01.120128 23008 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.2@KRBTEST.COM'} at 127.5.61.2:47425
WARNING: no policy specified for kudu/127.5.61.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
I20260809 20:21:01.164081 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:41859
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:21:01.502677 23402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:21:01.503000 23402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:21:01.503149 23402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:21:01.510339 23402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:21:01.510474 23402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:21:01.510634 23402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:21:01.520274 23402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:41859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:21:01.521519 23402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:21:01.522876 23402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:21:01.535771 23407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:21:01.535722 23408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:21:01.536195 23410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:21:01.536688 23402 server_base.cc:1034] running on GCE node
I20260809 20:21:01.537343 23402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:21:01.538338 23402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:21:01.539546 23402 hybrid_clock.cc:648] HybridClock initialized: now 1786306861539498 us; error 41 us; skew 500 ppm
I20260809 20:21:01.539896 23402 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:21:01 dist-test-slave-6k22 krb5kdc[22955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306861, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:21:01.542943 23402 init.cc:377] Logged in from keytab as kudu/127.5.61.3@KRBTEST.COM (short username kudu)
I20260809 20:21:01.544394 23402 webserver.cc:460] Webserver started at http://127.5.61.3:38339/ using document root <none> and password file <none>
I20260809 20:21:01.545068 23402 fs_manager.cc:359] Metadata directory not provided
I20260809 20:21:01.545208 23402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:21:01.545511 23402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:21:01.548525 23402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "f8eba6e4e390485b998ae5f0585d10d4"
format_stamp: "Formatted at 2026-08-09 20:21:01 on dist-test-slave-6k22"
server_key: "f50ff5258ed56bab4e582f3a940683db"
server_key_iv: "3859d0f4618d69f85b9b79f6ffc74f60"
server_key_version: "encryptionkey@0"
I20260809 20:21:01.549325 23402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "f8eba6e4e390485b998ae5f0585d10d4"
format_stamp: "Formatted at 2026-08-09 20:21:01 on dist-test-slave-6k22"
server_key: "f50ff5258ed56bab4e582f3a940683db"
server_key_iv: "3859d0f4618d69f85b9b79f6ffc74f60"
server_key_version: "encryptionkey@0"
I20260809 20:21:01.554315 23402 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 20:21:01.557583 23417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:01.558926 23402 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 20:21:01.559090 23402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "f8eba6e4e390485b998ae5f0585d10d4"
format_stamp: "Formatted at 2026-08-09 20:21:01 on dist-test-slave-6k22"
server_key: "f50ff5258ed56bab4e582f3a940683db"
server_key_iv: "3859d0f4618d69f85b9b79f6ffc74f60"
server_key_version: "encryptionkey@0"
I20260809 20:21:01.559295 23402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:21:01.592154 23242 heartbeater.cc:503] Master 127.5.61.62:41859 was elected leader, sending a full tablet report...
I20260809 20:21:01.604771 23402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:21:01.608749 23402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:21:01.609048 23402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:21:01.609969 23402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:21:01.611366 23402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:21:01.611469 23402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:01.611598 23402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:21:01.611662 23402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260809 20:21:01.645191 23402 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:40389
I20260809 20:21:01.645265 23542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:40389 every 8 connection(s)
I20260809 20:21:01.646512 23402 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:21:01.649940 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 23402
I20260809 20:21:01.650143 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:21:01.650748 5364 external_mini_cluster.cc:1428] Setting key df25df0fa4ff418164720510be2ca9f1
Aug 09 20:21:01 dist-test-slave-6k22 krb5kdc[22955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306861, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:21:01.667217 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:01.649157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:45969 (local address 127.5.61.62:41859)
0809 20:21:01.649601 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:01.649605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:01.650777 (+ 1172us) server_negotiation.cc:407] Connection header received
0809 20:21:01.652985 (+ 2208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:01.652991 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:01.653085 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:01.653197 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:21:01.654771 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.655527 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.657122 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.657574 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.657727 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:01.661265 (+ 3538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:21:01.661270 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:21:01.661273 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:21:01.661306 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:21:01.664183 (+ 2877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:01.665049 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:01.665053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:01.665055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:01.665170 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:01.665657 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:01.665661 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:01.665664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:01.666040 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:21:01.666185 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:01.666794 (+ 609us) server_negotiation.cc:299] Negotiation successful
0809 20:21:01.666965 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":225,"threads_started":1}
I20260809 20:21:01.668227 23547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:01.649815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.3:45969)
0809 20:21:01.650624 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:01.650664 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 20:21:01.652704 (+ 2040us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:21:01.653435 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:21:01.653444 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:21:01.654076 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:21:01.654607 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:01.654625 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.655719 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.655729 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:01.656971 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:01.656984 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.657831 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.657840 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:01.658078 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:01.658776 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:21:01.658798 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:21:01.661050 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:21:01.664326 (+ 3276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:21:01.664332 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:21:01.664342 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:21:01.664944 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:21:01.665288 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:21:01.665291 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:21:01.665293 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:21:01.665542 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:21:01.666176 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:21:01.666185 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:21:01.666493 (+ 308us) client_negotiation.cc:769] Sending connection context
0809 20:21:01.666715 (+ 222us) client_negotiation.cc:240] Negotiation successful
0809 20:21:01.667068 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":372,"threads_started":1}
I20260809 20:21:01.669502 23543 heartbeater.cc:347] Connected to a master server at 127.5.61.62:41859
I20260809 20:21:01.669799 23543 heartbeater.cc:464] Registering TS with master...
I20260809 20:21:01.670498 23543 heartbeater.cc:511] Master 127.5.61.62:41859 requested a full tablet report, sending...
I20260809 20:21:01.671898 23008 ts_manager.cc:194] Registered new tserver with Master: f8eba6e4e390485b998ae5f0585d10d4 (127.5.61.3:40389)
I20260809 20:21:01.672716 23008 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.3@KRBTEST.COM'} at 127.5.61.3:45969
I20260809 20:21:01.678093 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 20:21:01.695690 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:01.682497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49854 (local address 127.5.61.62:41859)
0809 20:21:01.682658 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:01.682663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:01.683120 (+ 457us) server_negotiation.cc:407] Connection header received
0809 20:21:01.683309 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:01.683314 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:01.683385 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:01.683481 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:21:01.684937 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.685654 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.687032 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.687460 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.687626 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:01.690603 (+ 2977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:21:01.690609 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:21:01.690613 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:21:01.690651 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:21:01.692959 (+ 2308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:01.693674 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:01.693680 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:01.693683 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:01.693824 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:01.694180 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:01.694186 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:01.694189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:01.694517 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:21:01.694772 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:01.695091 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 20:21:01.695330 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 20:21:01.699594 23008 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49854:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:21:01.706477 23008 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:21:01.732105 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:01.718335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:37737 (local address 127.0.0.1:48936)
0809 20:21:01.720044 (+ 1709us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:01.720108 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 20:21:01.720330 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:21:01.724555 (+ 4225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:21:01.724559 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:21:01.724584 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:21:01.725070 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:01.725085 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.727066 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.727074 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:01.729263 (+ 2189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:01.729279 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.730986 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.730992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:01.731194 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:01.731257 (+ 63us) client_negotiation.cc:769] Sending connection context
0809 20:21:01.731416 (+ 159us) client_negotiation.cc:240] Negotiation successful
0809 20:21:01.731639 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1408,"mutex_wait_us":65,"thread_start_us":526,"threads_started":1}
I20260809 20:21:01.732234 23558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:01.719110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:44147 (local address 127.0.0.1:59528)
0809 20:21:01.723673 (+ 4563us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:01.723705 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 20:21:01.723846 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:21:01.724798 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:21:01.724808 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:21:01.724834 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:21:01.725304 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:01.725315 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.726906 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.726918 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:01.729099 (+ 2181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:01.729111 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.731362 (+ 2251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.731368 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:01.731601 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:01.731632 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 20:21:01.731707 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 20:21:01.731838 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4400,"mutex_wait_us":154,"thread_start_us":1781,"threads_started":1}
I20260809 20:21:01.732772 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:01.719919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45856 (local address 127.5.61.3:40389)
0809 20:21:01.722054 (+ 2135us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:01.722066 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:01.722109 (+ 43us) server_negotiation.cc:407] Connection header received
0809 20:21:01.722247 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:01.722259 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:01.722557 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:01.724315 (+ 1758us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:21:01.724634 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.725814 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.729414 (+ 3600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.730848 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.730959 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:01.731178 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:01.732230 (+ 1052us) server_negotiation.cc:299] Negotiation successful
0809 20:21:01.732411 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1906,"thread_start_us":1707,"threads_started":1}
I20260809 20:21:01.732812 23563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:01.720619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59528 (local address 127.5.61.1:44147)
0809 20:21:01.723367 (+ 2748us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:01.723375 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:01.723992 (+ 617us) server_negotiation.cc:407] Connection header received
0809 20:21:01.724094 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:01.724101 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:01.724334 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:01.724528 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:21:01.725486 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.726768 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.729274 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.730776 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.730933 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:01.732258 (+ 1325us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:01.732327 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 20:21:01.732478 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2529,"thread_start_us":2347,"threads_started":1}
I20260809 20:21:01.733225 23561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:01.718609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:40389 (local address 127.0.0.1:45856)
0809 20:21:01.721056 (+ 2447us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:01.721075 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 20:21:01.721181 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:21:01.723526 (+ 2345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:21:01.723533 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:21:01.723595 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:21:01.724075 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:01.724089 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.725967 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.725972 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:01.729263 (+ 3291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:01.729279 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.731400 (+ 2121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.731411 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:01.731630 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:01.731671 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 20:21:01.731753 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 20:21:01.731889 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2285,"mutex_wait_us":651,"thread_start_us":4761,"threads_started":1}
I20260809 20:21:01.734127 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:01.718723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48936 (local address 127.5.61.2:37737)
0809 20:21:01.724004 (+ 5281us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:01.724014 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:01.724038 (+ 24us) server_negotiation.cc:407] Connection header received
0809 20:21:01.724119 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:01.724125 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:01.724331 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:01.725317 (+ 986us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:21:01.725543 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.726900 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.729472 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.730784 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.730945 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:01.731138 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:01.732704 (+ 1566us) server_negotiation.cc:299] Negotiation successful
0809 20:21:01.733808 (+ 1104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5093,"thread_start_us":5675,"threads_started":1}
I20260809 20:21:01.736678 23161 tablet_service.cc:1467] Processing CreateTablet for tablet ec47ce8e566146ac8a60a66fb4447862 (DEFAULT_TABLE table=test-table [id=a9f5315c42bc4cefa7deb9a3c86e38e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:21:01.736861 23469 tablet_service.cc:1467] Processing CreateTablet for tablet ec47ce8e566146ac8a60a66fb4447862 (DEFAULT_TABLE table=test-table [id=a9f5315c42bc4cefa7deb9a3c86e38e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:21:01.737493 23322 tablet_service.cc:1467] Processing CreateTablet for tablet ec47ce8e566146ac8a60a66fb4447862 (DEFAULT_TABLE table=test-table [id=a9f5315c42bc4cefa7deb9a3c86e38e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:21:01.738221 23161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec47ce8e566146ac8a60a66fb4447862. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:21:01.738318 23469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec47ce8e566146ac8a60a66fb4447862. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:21:01.738780 23322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec47ce8e566146ac8a60a66fb4447862. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:21:01.751318 23564 tablet_bootstrap.cc:492] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5: Bootstrap starting.
I20260809 20:21:01.753463 23566 tablet_bootstrap.cc:492] T ec47ce8e566146ac8a60a66fb4447862 P 17954d03dc0144dfafdfa2d9383e8027: Bootstrap starting.
I20260809 20:21:01.755038 23565 tablet_bootstrap.cc:492] T ec47ce8e566146ac8a60a66fb4447862 P f8eba6e4e390485b998ae5f0585d10d4: Bootstrap starting.
I20260809 20:21:01.755733 23564 tablet_bootstrap.cc:654] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5: Neither blocks nor log segments found. Creating new log.
I20260809 20:21:01.757189 23564 log.cc:824] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5: Log is configured to *not* fsync() on all Append() calls
I20260809 20:21:01.757267 23566 tablet_bootstrap.cc:654] T ec47ce8e566146ac8a60a66fb4447862 P 17954d03dc0144dfafdfa2d9383e8027: Neither blocks nor log segments found. Creating new log.
I20260809 20:21:01.757810 23565 tablet_bootstrap.cc:654] T ec47ce8e566146ac8a60a66fb4447862 P f8eba6e4e390485b998ae5f0585d10d4: Neither blocks nor log segments found. Creating new log.
I20260809 20:21:01.758651 23566 log.cc:824] T ec47ce8e566146ac8a60a66fb4447862 P 17954d03dc0144dfafdfa2d9383e8027: Log is configured to *not* fsync() on all Append() calls
I20260809 20:21:01.759079 23565 log.cc:824] T ec47ce8e566146ac8a60a66fb4447862 P f8eba6e4e390485b998ae5f0585d10d4: Log is configured to *not* fsync() on all Append() calls
I20260809 20:21:01.761291 23564 tablet_bootstrap.cc:492] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5: No bootstrap required, opened a new log
I20260809 20:21:01.761569 23564 ts_tablet_manager.cc:1397] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260809 20:21:01.761794 23566 tablet_bootstrap.cc:492] T ec47ce8e566146ac8a60a66fb4447862 P 17954d03dc0144dfafdfa2d9383e8027: No bootstrap required, opened a new log
I20260809 20:21:01.762085 23566 ts_tablet_manager.cc:1397] T ec47ce8e566146ac8a60a66fb4447862 P 17954d03dc0144dfafdfa2d9383e8027: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 20:21:01.762435 23565 tablet_bootstrap.cc:492] T ec47ce8e566146ac8a60a66fb4447862 P f8eba6e4e390485b998ae5f0585d10d4: No bootstrap required, opened a new log
I20260809 20:21:01.762718 23565 ts_tablet_manager.cc:1397] T ec47ce8e566146ac8a60a66fb4447862 P f8eba6e4e390485b998ae5f0585d10d4: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260809 20:21:01.769907 23564 raft_consensus.cc:348] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17954d03dc0144dfafdfa2d9383e8027" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37737 } } peers { permanent_uuid: "f8eba6e4e390485b998ae5f0585d10d4" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40389 } } peers { permanent_uuid: "8ebf53713a3143e581f3e40def807bb5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44147 } }
I20260809 20:21:01.770295 23564 raft_consensus.cc:374] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:21:01.770419 23564 raft_consensus.cc:729] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebf53713a3143e581f3e40def807bb5, State: Initialized, Role: FOLLOWER
I20260809 20:21:01.771044 23564 consensus_queue.cc:260] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17954d03dc0144dfafdfa2d9383e8027" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37737 } } peers { permanent_uuid: "f8eba6e4e390485b998ae5f0585d10d4" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40389 } } peers { permanent_uuid: "8ebf53713a3143e581f3e40def807bb5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44147 } }
I20260809 20:21:01.773003 23564 ts_tablet_manager.cc:1428] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5: Time spent starting tablet: real 0.011s user 0.006s sys 0.006s
I20260809 20:21:01.772775 23566 raft_consensus.cc:348] T ec47ce8e566146ac8a60a66fb4447862 P 17954d03dc0144dfafdfa2d9383e8027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17954d03dc0144dfafdfa2d9383e8027" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37737 } } peers { permanent_uuid: "f8eba6e4e390485b998ae5f0585d10d4" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40389 } } peers { permanent_uuid: "8ebf53713a3143e581f3e40def807bb5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44147 } }
I20260809 20:21:01.773223 23566 raft_consensus.cc:374] T ec47ce8e566146ac8a60a66fb4447862 P 17954d03dc0144dfafdfa2d9383e8027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:21:01.773327 23566 raft_consensus.cc:729] T ec47ce8e566146ac8a60a66fb4447862 P 17954d03dc0144dfafdfa2d9383e8027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17954d03dc0144dfafdfa2d9383e8027, State: Initialized, Role: FOLLOWER
I20260809 20:21:01.773299 23565 raft_consensus.cc:348] T ec47ce8e566146ac8a60a66fb4447862 P f8eba6e4e390485b998ae5f0585d10d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17954d03dc0144dfafdfa2d9383e8027" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37737 } } peers { permanent_uuid: "f8eba6e4e390485b998ae5f0585d10d4" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40389 } } peers { permanent_uuid: "8ebf53713a3143e581f3e40def807bb5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44147 } }
I20260809 20:21:01.773720 23565 raft_consensus.cc:374] T ec47ce8e566146ac8a60a66fb4447862 P f8eba6e4e390485b998ae5f0585d10d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:21:01.773820 23565 raft_consensus.cc:729] T ec47ce8e566146ac8a60a66fb4447862 P f8eba6e4e390485b998ae5f0585d10d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8eba6e4e390485b998ae5f0585d10d4, State: Initialized, Role: FOLLOWER
I20260809 20:21:01.774004 23566 consensus_queue.cc:260] T ec47ce8e566146ac8a60a66fb4447862 P 17954d03dc0144dfafdfa2d9383e8027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17954d03dc0144dfafdfa2d9383e8027" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37737 } } peers { permanent_uuid: "f8eba6e4e390485b998ae5f0585d10d4" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40389 } } peers { permanent_uuid: "8ebf53713a3143e581f3e40def807bb5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44147 } }
I20260809 20:21:01.774379 23565 consensus_queue.cc:260] T ec47ce8e566146ac8a60a66fb4447862 P f8eba6e4e390485b998ae5f0585d10d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17954d03dc0144dfafdfa2d9383e8027" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37737 } } peers { permanent_uuid: "f8eba6e4e390485b998ae5f0585d10d4" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40389 } } peers { permanent_uuid: "8ebf53713a3143e581f3e40def807bb5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44147 } }
I20260809 20:21:01.775243 23395 heartbeater.cc:503] Master 127.5.61.62:41859 was elected leader, sending a full tablet report...
I20260809 20:21:01.776081 23566 ts_tablet_manager.cc:1428] T ec47ce8e566146ac8a60a66fb4447862 P 17954d03dc0144dfafdfa2d9383e8027: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260809 20:21:01.778084 23543 heartbeater.cc:503] Master 127.5.61.62:41859 was elected leader, sending a full tablet report...
I20260809 20:21:01.778764 23565 ts_tablet_manager.cc:1428] T ec47ce8e566146ac8a60a66fb4447862 P f8eba6e4e390485b998ae5f0585d10d4: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260809 20:21:01.793553 23570 raft_consensus.cc:482] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:21:01.793825 23570 raft_consensus.cc:504] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17954d03dc0144dfafdfa2d9383e8027" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37737 } } peers { permanent_uuid: "f8eba6e4e390485b998ae5f0585d10d4" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40389 } } peers { permanent_uuid: "8ebf53713a3143e581f3e40def807bb5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44147 } }
I20260809 20:21:01.795794 23570 leader_election.cc:288] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17954d03dc0144dfafdfa2d9383e8027 (127.5.61.2:37737), f8eba6e4e390485b998ae5f0585d10d4 (127.5.61.3:40389)
I20260809 20:21:01.803720 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:01.796149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:38045 (local address 127.5.61.2:37737)
0809 20:21:01.796329 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:01.796335 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:01.796935 (+ 600us) server_negotiation.cc:407] Connection header received
0809 20:21:01.797165 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:01.797170 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:01.797226 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:01.797325 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:21:01.797962 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.798966 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.801520 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.802712 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.802847 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:01.802947 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:01.803324 (+ 377us) server_negotiation.cc:299] Negotiation successful
0809 20:21:01.803425 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 20:21:01.803948 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:01.796408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:40161 (local address 127.5.61.3:40389)
0809 20:21:01.796592 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:01.796600 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:01.797301 (+ 701us) server_negotiation.cc:407] Connection header received
0809 20:21:01.797561 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:01.797565 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:01.797619 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:01.797697 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:21:01.798502 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.799380 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.801756 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.802966 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.803113 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:01.803183 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:01.803588 (+ 405us) server_negotiation.cc:299] Negotiation successful
0809 20:21:01.803708 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 20:21:01.803990 23574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:01.795972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:37737 (local address 127.5.61.1:38045)
0809 20:21:01.796815 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:01.796838 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 20:21:01.797018 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:21:01.797395 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:21:01.797399 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:21:01.797424 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:21:01.797817 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:01.797827 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.799088 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.799092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:01.801382 (+ 2290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:01.801396 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.802931 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.802935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:01.803139 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:01.803184 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 20:21:01.803287 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 20:21:01.803415 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"mutex_wait_us":71,"thread_start_us":739,"threads_started":1}
I20260809 20:21:01.804057 23575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:01.796435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:40389 (local address 127.5.61.1:40161)
0809 20:21:01.797202 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:01.797217 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:21:01.797368 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:21:01.797903 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:21:01.797907 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:21:01.797935 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:21:01.798276 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:01.798289 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.799513 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.799518 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:01.801628 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:01.801637 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.803233 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.803242 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:01.803401 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:01.803429 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 20:21:01.803515 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 20:21:01.803676 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":3,"thread_start_us":516,"threads_started":1}
I20260809 20:21:01.805289 23490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec47ce8e566146ac8a60a66fb4447862" candidate_uuid: "8ebf53713a3143e581f3e40def807bb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8eba6e4e390485b998ae5f0585d10d4" is_pre_election: true
I20260809 20:21:01.805289 23344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec47ce8e566146ac8a60a66fb4447862" candidate_uuid: "8ebf53713a3143e581f3e40def807bb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17954d03dc0144dfafdfa2d9383e8027" is_pre_election: true
I20260809 20:21:01.805858 23344 raft_consensus.cc:2454] T ec47ce8e566146ac8a60a66fb4447862 P 17954d03dc0144dfafdfa2d9383e8027 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebf53713a3143e581f3e40def807bb5 in term 0.
I20260809 20:21:01.805858 23490 raft_consensus.cc:2454] T ec47ce8e566146ac8a60a66fb4447862 P f8eba6e4e390485b998ae5f0585d10d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebf53713a3143e581f3e40def807bb5 in term 0.
I20260809 20:21:01.806537 23100 leader_election.cc:302] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17954d03dc0144dfafdfa2d9383e8027, 8ebf53713a3143e581f3e40def807bb5; no voters:
I20260809 20:21:01.806996 23570 raft_consensus.cc:2781] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:21:01.807142 23570 raft_consensus.cc:482] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:21:01.807276 23570 raft_consensus.cc:3037] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:21:01.809563 23570 raft_consensus.cc:504] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17954d03dc0144dfafdfa2d9383e8027" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37737 } } peers { permanent_uuid: "f8eba6e4e390485b998ae5f0585d10d4" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40389 } } peers { permanent_uuid: "8ebf53713a3143e581f3e40def807bb5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44147 } }
I20260809 20:21:01.810144 23570 leader_election.cc:288] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5 [CANDIDATE]: Term 1 election: Requested vote from peers 17954d03dc0144dfafdfa2d9383e8027 (127.5.61.2:37737), f8eba6e4e390485b998ae5f0585d10d4 (127.5.61.3:40389)
I20260809 20:21:01.810643 23344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec47ce8e566146ac8a60a66fb4447862" candidate_uuid: "8ebf53713a3143e581f3e40def807bb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17954d03dc0144dfafdfa2d9383e8027"
I20260809 20:21:01.810643 23490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec47ce8e566146ac8a60a66fb4447862" candidate_uuid: "8ebf53713a3143e581f3e40def807bb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8eba6e4e390485b998ae5f0585d10d4"
I20260809 20:21:01.810846 23490 raft_consensus.cc:3037] T ec47ce8e566146ac8a60a66fb4447862 P f8eba6e4e390485b998ae5f0585d10d4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:21:01.810846 23344 raft_consensus.cc:3037] T ec47ce8e566146ac8a60a66fb4447862 P 17954d03dc0144dfafdfa2d9383e8027 [term 0 FOLLOWER]: Advancing to term 1
W20260809 20:21:01.813370 23243 tablet.cc:2369] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:21:01.814148 23344 raft_consensus.cc:2454] T ec47ce8e566146ac8a60a66fb4447862 P 17954d03dc0144dfafdfa2d9383e8027 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebf53713a3143e581f3e40def807bb5 in term 1.
I20260809 20:21:01.814148 23490 raft_consensus.cc:2454] T ec47ce8e566146ac8a60a66fb4447862 P f8eba6e4e390485b998ae5f0585d10d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebf53713a3143e581f3e40def807bb5 in term 1.
I20260809 20:21:01.814556 23100 leader_election.cc:302] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 17954d03dc0144dfafdfa2d9383e8027, 8ebf53713a3143e581f3e40def807bb5, f8eba6e4e390485b998ae5f0585d10d4; no voters:
I20260809 20:21:01.814946 23570 raft_consensus.cc:2781] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:21:01.815335 23570 raft_consensus.cc:686] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5 [term 1 LEADER]: Becoming Leader. State: Replica: 8ebf53713a3143e581f3e40def807bb5, State: Running, Role: LEADER
I20260809 20:21:01.815828 23570 consensus_queue.cc:237] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17954d03dc0144dfafdfa2d9383e8027" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37737 } } peers { permanent_uuid: "f8eba6e4e390485b998ae5f0585d10d4" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40389 } } peers { permanent_uuid: "8ebf53713a3143e581f3e40def807bb5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44147 } }
I20260809 20:21:01.820366 23008 catalog_manager.cc:5495] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ebf53713a3143e581f3e40def807bb5 (127.5.61.1). New cstate: current_term: 1 leader_uuid: "8ebf53713a3143e581f3e40def807bb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17954d03dc0144dfafdfa2d9383e8027" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8eba6e4e390485b998ae5f0585d10d4" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebf53713a3143e581f3e40def807bb5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44147 } health_report { overall_health: HEALTHY } } }
W20260809 20:21:01.845134 23396 tablet.cc:2369] T ec47ce8e566146ac8a60a66fb4447862 P 17954d03dc0144dfafdfa2d9383e8027: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:21:01.853479 23563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:01.848359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59536 (local address 127.5.61.1:44147)
0809 20:21:01.848565 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:01.848570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:01.848608 (+ 38us) server_negotiation.cc:407] Connection header received
0809 20:21:01.848778 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:01.848782 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:01.848907 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:01.849021 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:21:01.849380 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.850091 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.851456 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.851904 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.852001 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:01.852460 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:21:01.852945 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:21:01.853051 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:01.853133 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 20:21:01.853195 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 20:21:01.861315 23344 raft_consensus.cc:1261] T ec47ce8e566146ac8a60a66fb4447862 P 17954d03dc0144dfafdfa2d9383e8027 [term 1 FOLLOWER]: Refusing update from remote peer 8ebf53713a3143e581f3e40def807bb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:21:01.861356 23490 raft_consensus.cc:1261] T ec47ce8e566146ac8a60a66fb4447862 P f8eba6e4e390485b998ae5f0585d10d4 [term 1 FOLLOWER]: Refusing update from remote peer 8ebf53713a3143e581f3e40def807bb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:21:01.862278 23577 consensus_queue.cc:1035] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17954d03dc0144dfafdfa2d9383e8027" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:21:01.862649 23570 consensus_queue.cc:1035] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8eba6e4e390485b998ae5f0585d10d4" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:21:01.872242 23580 mvcc.cc:205] Tried to move back new op lower bound from 7316712906174361600 to 7316712906001428480. Current Snapshot: MvccSnapshot[applied={T|T < 7316712906174361600}]
I20260809 20:21:01.875907 23582 mvcc.cc:205] Tried to move back new op lower bound from 7316712906174361600 to 7316712906001428480. Current Snapshot: MvccSnapshot[applied={T|T < 7316712906174361600}]
I20260809 20:21:01.880798 23581 mvcc.cc:205] Tried to move back new op lower bound from 7316712906174361600 to 7316712906001428480. Current Snapshot: MvccSnapshot[applied={T|T < 7316712906174361600}]
I20260809 20:21:01.890367 23007 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:21:01.890658 23007 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:21:01.894676 23007 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906: Sending DeleteTablet for 3 replicas of tablet ec47ce8e566146ac8a60a66fb4447862
I20260809 20:21:01.895922 23322 tablet_service.cc:1514] Processing DeleteTablet for tablet ec47ce8e566146ac8a60a66fb4447862 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:21:01 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:48936
I20260809 20:21:01.896119 23161 tablet_service.cc:1514] Processing DeleteTablet for tablet ec47ce8e566146ac8a60a66fb4447862 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:21:01 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:59528
I20260809 20:21:01.897193 23469 tablet_service.cc:1514] Processing DeleteTablet for tablet ec47ce8e566146ac8a60a66fb4447862 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:21:01 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:45856
I20260809 20:21:01.902858 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 23082
I20260809 20:21:01.908100 23590 tablet_replica.cc:330] T ec47ce8e566146ac8a60a66fb4447862 P 17954d03dc0144dfafdfa2d9383e8027: stopping tablet replica
I20260809 20:21:01.908475 23590 raft_consensus.cc:2229] T ec47ce8e566146ac8a60a66fb4447862 P 17954d03dc0144dfafdfa2d9383e8027 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:21:01.908939 23590 raft_consensus.cc:2258] T ec47ce8e566146ac8a60a66fb4447862 P 17954d03dc0144dfafdfa2d9383e8027 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:21:01.910817 23590 ts_tablet_manager.cc:1905] T ec47ce8e566146ac8a60a66fb4447862 P 17954d03dc0144dfafdfa2d9383e8027: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:21:01.915748 23590 ts_tablet_manager.cc:1918] T ec47ce8e566146ac8a60a66fb4447862 P 17954d03dc0144dfafdfa2d9383e8027: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:21:01.915920 23590 log.cc:1196] T ec47ce8e566146ac8a60a66fb4447862 P 17954d03dc0144dfafdfa2d9383e8027: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/ec47ce8e566146ac8a60a66fb4447862
I20260809 20:21:01.916476 23590 ts_tablet_manager.cc:1939] T ec47ce8e566146ac8a60a66fb4447862 P 17954d03dc0144dfafdfa2d9383e8027: Deleting consensus metadata
I20260809 20:21:01.917089 23592 tablet_replica.cc:330] T ec47ce8e566146ac8a60a66fb4447862 P f8eba6e4e390485b998ae5f0585d10d4: stopping tablet replica
I20260809 20:21:01.917670 23592 raft_consensus.cc:2229] T ec47ce8e566146ac8a60a66fb4447862 P f8eba6e4e390485b998ae5f0585d10d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:21:01.918313 23591 tablet_replica.cc:330] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5: stopping tablet replica
I20260809 20:21:01.918474 22988 catalog_manager.cc:4841] TS 17954d03dc0144dfafdfa2d9383e8027 (127.5.61.2:37737): tablet ec47ce8e566146ac8a60a66fb4447862 (table test-table [id=a9f5315c42bc4cefa7deb9a3c86e38e6]) successfully deleted
I20260809 20:21:01.918727 23591 raft_consensus.cc:2229] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:21:01.919305 23591 raft_consensus.cc:2258] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:21:01.919997 23592 raft_consensus.cc:2258] T ec47ce8e566146ac8a60a66fb4447862 P f8eba6e4e390485b998ae5f0585d10d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:21:01.921806 23592 ts_tablet_manager.cc:1905] T ec47ce8e566146ac8a60a66fb4447862 P f8eba6e4e390485b998ae5f0585d10d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:21:01.926990 23592 ts_tablet_manager.cc:1918] T ec47ce8e566146ac8a60a66fb4447862 P f8eba6e4e390485b998ae5f0585d10d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:21:01.927163 23592 log.cc:1196] T ec47ce8e566146ac8a60a66fb4447862 P f8eba6e4e390485b998ae5f0585d10d4: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/ec47ce8e566146ac8a60a66fb4447862
I20260809 20:21:01.927165 23591 ts_tablet_manager.cc:1905] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:21:01.927757 23592 ts_tablet_manager.cc:1939] T ec47ce8e566146ac8a60a66fb4447862 P f8eba6e4e390485b998ae5f0585d10d4: Deleting consensus metadata
Aug 09 20:21:01 dist-test-slave-6k22 krb5kdc[22955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.1@KRBTEST.COM
I20260809 20:21:01.929545 22989 catalog_manager.cc:4841] TS f8eba6e4e390485b998ae5f0585d10d4 (127.5.61.3:40389): tablet ec47ce8e566146ac8a60a66fb4447862 (table test-table [id=a9f5315c42bc4cefa7deb9a3c86e38e6]) successfully deleted
I20260809 20:21:01.935000 23591 ts_tablet_manager.cc:1918] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:21:01.935214 23591 log.cc:1196] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/ec47ce8e566146ac8a60a66fb4447862
I20260809 20:21:01.935762 23591 ts_tablet_manager.cc:1939] T ec47ce8e566146ac8a60a66fb4447862 P 8ebf53713a3143e581f3e40def807bb5: Deleting consensus metadata
I20260809 20:21:01.937528 22991 catalog_manager.cc:4841] TS 8ebf53713a3143e581f3e40def807bb5 (127.5.61.1:44147): tablet ec47ce8e566146ac8a60a66fb4447862 (table test-table [id=a9f5315c42bc4cefa7deb9a3c86e38e6]) successfully deleted
I20260809 20:21:01.938933 23563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:01.904054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59550 (local address 127.5.61.1:44147)
0809 20:21:01.904225 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:01.904254 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:01.916390 (+ 12136us) server_negotiation.cc:407] Connection header received
0809 20:21:01.917325 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:01.917331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:01.917428 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:01.917532 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:21:01.920366 (+ 2834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.921525 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.923788 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:01.924515 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:01.924616 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:01.930656 (+ 6040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:21:01.930670 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:21:01.930688 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:21:01.930730 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:21:01.935005 (+ 4275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:01.935962 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:01.935972 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:01.935975 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:01.936186 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:01.936730 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:01.936735 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:01.936738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:01.937200 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:21:01.937575 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:01.938273 (+ 698us) server_negotiation.cc:299] Negotiation successful
0809 20:21:01.938457 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 20:21:01.939688 23237 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:21:02.040071 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 23082
I20260809 20:21:02.062244 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 23249
Aug 09 20:21:02 dist-test-slave-6k22 krb5kdc[22955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.2@KRBTEST.COM
I20260809 20:21:02.077500 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:02.063077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48942 (local address 127.5.61.2:37737)
0809 20:21:02.063246 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:02.063251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:02.063269 (+ 18us) server_negotiation.cc:407] Connection header received
0809 20:21:02.063406 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:02.063409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:02.063471 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:02.063550 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:21:02.064982 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:02.065728 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:02.067065 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:02.067451 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:02.067548 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:02.072085 (+ 4537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:21:02.072097 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:21:02.072112 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:21:02.072145 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:21:02.074868 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:02.075512 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:02.075519 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:02.075522 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:02.075730 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:02.076097 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:02.076100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:02.076102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:02.076467 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:21:02.076693 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:02.076964 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 20:21:02.077126 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 20:21:02.078213 23390 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:21:02.166939 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 23249
I20260809 20:21:02.192082 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 23402
Aug 09 20:21:02 dist-test-slave-6k22 krb5kdc[22955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.3@KRBTEST.COM
I20260809 20:21:02.207077 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:02.192940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45872 (local address 127.5.61.3:40389)
0809 20:21:02.193129 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:02.193134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:02.193152 (+ 18us) server_negotiation.cc:407] Connection header received
0809 20:21:02.193263 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:02.193267 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:02.193332 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:02.193418 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:21:02.194692 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:02.195439 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:02.197021 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:02.197410 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:02.197512 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:02.201901 (+ 4389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:21:02.201914 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:21:02.201930 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:21:02.201958 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:21:02.204531 (+ 2573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:02.205243 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:02.205249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:02.205251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:02.205379 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:02.205693 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:02.205696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:02.205698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:02.206051 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:21:02.206276 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:02.206527 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 20:21:02.206704 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 20:21:02.207787 23537 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:21:02.296911 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 23402
I20260809 20:21:02.318066 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 22970
I20260809 20:21:02.319227 23043 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:21:02.430851 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 22970
I20260809 20:21:22.456512 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:41859
--webserver_interface=127.5.61.62
--webserver_port=32847
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:41859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:21:22.791256 23598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:21:22.791530 23598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:21:22.791687 23598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:21:22.798712 23598 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 20:21:22.798796 23598 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 20:21:22.798835 23598 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 20:21:22.798889 23598 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:21:22.798942 23598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:21:22.799010 23598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:21:22.799055 23598 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:21:22.799109 23598 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:21:22.808161 23598 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:41859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:41859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=32847
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:21:22.809279 23598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:21:22.810487 23598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:21:22.833747 23605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:21:22.833757 23606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:21:22.833814 23608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:21:22.834622 23598 server_base.cc:1034] running on GCE node
I20260809 20:21:22.835481 23598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:21:22.837036 23598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:21:22.838292 23598 hybrid_clock.cc:648] HybridClock initialized: now 1786306882838236 us; error 54 us; skew 500 ppm
I20260809 20:21:22.838727 23598 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:21:22 dist-test-slave-6k22 krb5kdc[22955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306882, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:21:22.842402 23598 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:21:22.844018 23598 webserver.cc:460] Webserver started at http://127.5.61.62:32847/ using document root <none> and password file <none>
I20260809 20:21:22.844815 23598 fs_manager.cc:359] Metadata directory not provided
I20260809 20:21:22.844975 23598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:21:22.850905 23598 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 20:21:22.853945 23619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:22.855342 23598 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 20:21:22.855540 23598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "8ef1eae90ca245239d5944f2d0e0d906"
format_stamp: "Formatted at 2026-08-09 20:20:59 on dist-test-slave-6k22"
server_key: "cb7a94fcb2a7d1a27eb7c976fca49b0e"
server_key_iv: "b069ff8c22cc92a1c33d495ff4cca014"
server_key_version: "encryptionkey@0"
I20260809 20:21:22.856696 23598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:21:22.885120 23598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:21:22.889765 23598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:21:22.890092 23598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:21:22.909466 23598 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:41859
I20260809 20:21:22.909502 23679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:41859 every 8 connection(s)
I20260809 20:21:22.911032 23598 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:21:22.915622 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 23598
I20260809 20:21:22.916510 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:44147
--local_ip_for_outbound_sockets=127.5.61.1
--tserver_master_addrs=127.5.61.62:41859
--webserver_port=40405
--webserver_interface=127.5.61.1
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 20:21:22.918138 23682 sys_catalog.cc:263] Verifying existing consensus state
I20260809 20:21:22.921437 23682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906: Bootstrap starting.
I20260809 20:21:22.941447 23682 log.cc:824] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906: Log is configured to *not* fsync() on all Append() calls
I20260809 20:21:22.953433 23682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 20:21:22.953998 23682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906: Bootstrap complete.
I20260809 20:21:22.963404 23682 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef1eae90ca245239d5944f2d0e0d906" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41859 } }
I20260809 20:21:22.964329 23682 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef1eae90ca245239d5944f2d0e0d906, State: Initialized, Role: FOLLOWER
I20260809 20:21:22.964942 23682 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef1eae90ca245239d5944f2d0e0d906" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41859 } }
I20260809 20:21:22.965219 23682 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:21:22.965364 23682 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:21:22.965550 23682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 1 FOLLOWER]: Advancing to term 2
I20260809 20:21:22.969223 23682 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef1eae90ca245239d5944f2d0e0d906" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41859 } }
I20260809 20:21:22.969795 23682 leader_election.cc:302] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ef1eae90ca245239d5944f2d0e0d906; no voters:
I20260809 20:21:22.970503 23682 leader_election.cc:288] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 20:21:22.970640 23687 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 20:21:22.972081 23687 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 2 LEADER]: Becoming Leader. State: Replica: 8ef1eae90ca245239d5944f2d0e0d906, State: Running, Role: LEADER
I20260809 20:21:22.972661 23687 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef1eae90ca245239d5944f2d0e0d906" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41859 } }
I20260809 20:21:22.972990 23682 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:21:22.975730 23689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8ef1eae90ca245239d5944f2d0e0d906" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef1eae90ca245239d5944f2d0e0d906" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41859 } } }
I20260809 20:21:22.976063 23689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [sys.catalog]: This master's current role is: LEADER
I20260809 20:21:22.975251 23690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ef1eae90ca245239d5944f2d0e0d906. Latest consensus state: current_term: 2 leader_uuid: "8ef1eae90ca245239d5944f2d0e0d906" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef1eae90ca245239d5944f2d0e0d906" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41859 } } }
I20260809 20:21:22.976644 23690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [sys.catalog]: This master's current role is: LEADER
I20260809 20:21:22.985313 23694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:21:22.995127 23694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:21:22.996500 23694 catalog_manager.cc:1259] Loaded cluster ID: 67627d0ed1f84966b661aed141f857a7
I20260809 20:21:22.996605 23694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:21:23.000591 23694 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:21:23.002115 23694 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906: Loaded TSK: 0
I20260809 20:21:23.044226 23694 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906: Generated new TSK 1
I20260809 20:21:23.044654 23694 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 20:21:23.264541 23684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:21:23.264839 23684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:21:23.265013 23684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:21:23.272059 23684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:21:23.272198 23684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:21:23.272356 23684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:21:23.282171 23684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:44147
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=40405
--tserver_master_addrs=127.5.61.62:41859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:21:23.283486 23684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:21:23.284793 23684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:21:23.300383 23716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:21:23.300349 23717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:21:23.301034 23720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:21:23.301344 23684 server_base.cc:1034] running on GCE node
I20260809 20:21:23.301949 23684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:21:23.303000 23684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:21:23.304208 23684 hybrid_clock.cc:648] HybridClock initialized: now 1786306883304170 us; error 33 us; skew 500 ppm
I20260809 20:21:23.304529 23684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:21:23 dist-test-slave-6k22 krb5kdc[22955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306883, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:21:23.307973 23684 init.cc:377] Logged in from keytab as kudu/127.5.61.1@KRBTEST.COM (short username kudu)
I20260809 20:21:23.309620 23684 webserver.cc:460] Webserver started at http://127.5.61.1:40405/ using document root <none> and password file <none>
I20260809 20:21:23.310300 23684 fs_manager.cc:359] Metadata directory not provided
I20260809 20:21:23.310467 23684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:21:23.315932 23684 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:21:23.318818 23728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:23.320261 23684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:21:23.320417 23684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "8ebf53713a3143e581f3e40def807bb5"
format_stamp: "Formatted at 2026-08-09 20:21:00 on dist-test-slave-6k22"
server_key: "5f4f6ce7ff00d0d10209fa47881a28f6"
server_key_iv: "b2ffe81e908d47d898b5f33299bd776f"
server_key_version: "encryptionkey@0"
I20260809 20:21:23.321341 23684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:21:23.352273 23684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:21:23.356606 23684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:21:23.356931 23684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:21:23.358007 23684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:21:23.359362 23684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:21:23.359459 23684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:23.359632 23684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:21:23.359733 23684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:23.400147 23684 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:44147
I20260809 20:21:23.400244 23860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:44147 every 8 connection(s)
I20260809 20:21:23.401659 23684 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:21:23.405628 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 23684
I20260809 20:21:23.406333 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:37737
--local_ip_for_outbound_sockets=127.5.61.2
--tserver_master_addrs=127.5.61.62:41859
--webserver_port=42803
--webserver_interface=127.5.61.2
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 20:21:23 dist-test-slave-6k22 krb5kdc[22955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306883, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:21:23.427999 23866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:23.404965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.1:56093)
0809 20:21:23.405900 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:23.405957 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 20:21:23.407887 (+ 1930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:21:23.409387 (+ 1500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:21:23.409394 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:21:23.409883 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:21:23.410430 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:23.410447 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:23.412808 (+ 2361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:23.412812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:23.413993 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:23.414005 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:23.415603 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:23.415611 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:23.415807 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:23.416516 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:21:23.416547 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:21:23.418956 (+ 2409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:21:23.423806 (+ 4850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:21:23.423813 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:21:23.423825 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:21:23.424243 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:21:23.424782 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:21:23.424785 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:21:23.424787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:21:23.424987 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:21:23.425958 (+ 971us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:21:23.425963 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:21:23.426247 (+ 284us) client_negotiation.cc:769] Sending connection context
0809 20:21:23.426477 (+ 230us) client_negotiation.cc:240] Negotiation successful
0809 20:21:23.426886 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":411,"threads_started":1}
I20260809 20:21:23.429028 23865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:23.404706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:56093 (local address 127.5.61.62:41859)
0809 20:21:23.405470 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:23.405488 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:23.406100 (+ 612us) server_negotiation.cc:407] Connection header received
0809 20:21:23.408438 (+ 2338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:23.408458 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:23.408898 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:23.409277 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:21:23.410642 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:23.412621 (+ 1979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:23.414167 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:23.415312 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:23.415501 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:23.419230 (+ 3729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:21:23.419242 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:21:23.419263 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:21:23.419322 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:21:23.423667 (+ 4345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:23.424407 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:23.424418 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:23.424429 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:23.424661 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:23.425087 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:23.425095 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:23.425102 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:23.425803 (+ 701us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:21:23.426007 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:23.426752 (+ 745us) server_negotiation.cc:299] Negotiation successful
0809 20:21:23.427263 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":317,"threads_started":1}
I20260809 20:21:23.431900 23863 heartbeater.cc:347] Connected to a master server at 127.5.61.62:41859
I20260809 20:21:23.432195 23863 heartbeater.cc:464] Registering TS with master...
I20260809 20:21:23.432998 23863 heartbeater.cc:511] Master 127.5.61.62:41859 requested a full tablet report, sending...
I20260809 20:21:23.435865 23640 ts_manager.cc:194] Registered new tserver with Master: 8ebf53713a3143e581f3e40def807bb5 (127.5.61.1:44147)
I20260809 20:21:23.437520 23640 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.1@KRBTEST.COM'} at 127.5.61.1:56093
W20260809 20:21:23.776552 23867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:21:23.776894 23867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:21:23.777045 23867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:21:23.784448 23867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:21:23.784545 23867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:21:23.784737 23867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:21:23.794437 23867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:37737
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=42803
--tserver_master_addrs=127.5.61.62:41859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:21:23.795825 23867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:21:23.797084 23867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:21:23.810555 23874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:21:23.810667 23878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:21:23.810674 23872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:21:23.811518 23867 server_base.cc:1034] running on GCE node
I20260809 20:21:23.812196 23867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:21:23.813014 23867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:21:23.814244 23867 hybrid_clock.cc:648] HybridClock initialized: now 1786306883814190 us; error 60 us; skew 500 ppm
I20260809 20:21:23.814601 23867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:21:23 dist-test-slave-6k22 krb5kdc[22955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306883, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:21:23.817893 23867 init.cc:377] Logged in from keytab as kudu/127.5.61.2@KRBTEST.COM (short username kudu)
I20260809 20:21:23.819337 23867 webserver.cc:460] Webserver started at http://127.5.61.2:42803/ using document root <none> and password file <none>
I20260809 20:21:23.820055 23867 fs_manager.cc:359] Metadata directory not provided
I20260809 20:21:23.820173 23867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:21:23.825429 23867 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 20:21:23.828171 23886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:23.829644 23867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:21:23.829825 23867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "17954d03dc0144dfafdfa2d9383e8027"
format_stamp: "Formatted at 2026-08-09 20:21:01 on dist-test-slave-6k22"
server_key: "51d27605186ccefec7fe380c09ea5db5"
server_key_iv: "c28426eb1bab84ba6634008194d22c9d"
server_key_version: "encryptionkey@0"
I20260809 20:21:23.830801 23867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:21:23.856371 23867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:21:23.860981 23867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:21:23.861320 23867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:21:23.862344 23867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:21:23.863754 23867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:21:23.863880 23867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:23.864014 23867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:21:23.864120 23867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:23.898424 23867 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:37737
I20260809 20:21:23.898486 24013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:37737 every 8 connection(s)
I20260809 20:21:23.900020 23867 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:21:23.901607 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 23867
I20260809 20:21:23.902256 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:40389
--local_ip_for_outbound_sockets=127.5.61.3
--tserver_master_addrs=127.5.61.62:41859
--webserver_port=38339
--webserver_interface=127.5.61.3
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 20:21:23 dist-test-slave-6k22 krb5kdc[22955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306883, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:21:23.924810 23865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:23.902646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:37401 (local address 127.5.61.62:41859)
0809 20:21:23.902846 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:23.902852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:23.904280 (+ 1428us) server_negotiation.cc:407] Connection header received
0809 20:21:23.906142 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:23.906151 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:23.906271 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:23.906399 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:21:23.907901 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:23.909154 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:23.911585 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:23.912373 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:23.912527 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:23.916758 (+ 4231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:21:23.916764 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:21:23.916769 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:21:23.916810 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:21:23.920339 (+ 3529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:23.921378 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:23.921383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:23.921387 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:23.921520 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:23.922072 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:23.922080 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:23.922086 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:23.922591 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:21:23.922827 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:23.924180 (+ 1353us) server_negotiation.cc:299] Negotiation successful
0809 20:21:23.924427 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 20:21:23.926611 24018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:23.903125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.2:37401)
0809 20:21:23.904087 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:23.904134 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 20:21:23.905848 (+ 1714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:21:23.906618 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:21:23.906626 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:21:23.907107 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:21:23.907726 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:23.907745 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:23.909351 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:23.909366 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:23.911398 (+ 2032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:23.911412 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:23.912614 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:23.912621 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:23.912890 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:23.913699 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:21:23.913724 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:21:23.916503 (+ 2779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:21:23.920503 (+ 4000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:21:23.920516 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:21:23.920533 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:21:23.921238 (+ 705us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:21:23.921640 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:21:23.921645 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:21:23.921649 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:21:23.921909 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:21:23.922757 (+ 848us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:21:23.922765 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:21:23.923166 (+ 401us) client_negotiation.cc:769] Sending connection context
0809 20:21:23.923506 (+ 340us) client_negotiation.cc:240] Negotiation successful
0809 20:21:23.924735 (+ 1229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":446,"threads_started":1}
I20260809 20:21:23.928179 24014 heartbeater.cc:347] Connected to a master server at 127.5.61.62:41859
I20260809 20:21:23.928556 24014 heartbeater.cc:464] Registering TS with master...
I20260809 20:21:23.929533 24014 heartbeater.cc:511] Master 127.5.61.62:41859 requested a full tablet report, sending...
I20260809 20:21:23.931246 23640 ts_manager.cc:194] Registered new tserver with Master: 17954d03dc0144dfafdfa2d9383e8027 (127.5.61.2:37737)
I20260809 20:21:23.932112 23640 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.2@KRBTEST.COM'} at 127.5.61.2:37401
W20260809 20:21:24.245252 24021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:21:24.245584 24021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:21:24.245754 24021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:21:24.252815 24021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:21:24.252949 24021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:21:24.253119 24021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:21:24.262607 24021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:40389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=38339
--tserver_master_addrs=127.5.61.62:41859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:21:24.263906 24021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:21:24.265187 24021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:21:24.278205 24027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:21:24.278205 24032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:21:24.278242 24030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:21:24.279354 24021 server_base.cc:1034] running on GCE node
I20260809 20:21:24.280009 24021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:21:24.280946 24021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:21:24.282181 24021 hybrid_clock.cc:648] HybridClock initialized: now 1786306884282134 us; error 38 us; skew 500 ppm
I20260809 20:21:24.282518 24021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:21:24 dist-test-slave-6k22 krb5kdc[22955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306884, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:21:24.286095 24021 init.cc:377] Logged in from keytab as kudu/127.5.61.3@KRBTEST.COM (short username kudu)
I20260809 20:21:24.287689 24021 webserver.cc:460] Webserver started at http://127.5.61.3:38339/ using document root <none> and password file <none>
I20260809 20:21:24.288312 24021 fs_manager.cc:359] Metadata directory not provided
I20260809 20:21:24.288427 24021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:21:24.293761 24021 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 20:21:24.298496 24042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:24.299872 24021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 20:21:24.300053 24021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "f8eba6e4e390485b998ae5f0585d10d4"
format_stamp: "Formatted at 2026-08-09 20:21:01 on dist-test-slave-6k22"
server_key: "f50ff5258ed56bab4e582f3a940683db"
server_key_iv: "3859d0f4618d69f85b9b79f6ffc74f60"
server_key_version: "encryptionkey@0"
I20260809 20:21:24.301020 24021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:21:24.326387 24021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:21:24.330437 24021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:21:24.330714 24021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:21:24.331745 24021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:21:24.333060 24021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:21:24.333174 24021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:24.333305 24021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:21:24.333397 24021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:24.366979 24021 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:40389
I20260809 20:21:24.367096 24172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:40389 every 8 connection(s)
I20260809 20:21:24.368445 24021 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:21:24.374769 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 24021
Aug 09 20:21:24 dist-test-slave-6k22 krb5kdc[22955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306884, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:21:24.391811 23865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:24.371066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:39605 (local address 127.5.61.62:41859)
0809 20:21:24.371255 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:24.371260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:24.372459 (+ 1199us) server_negotiation.cc:407] Connection header received
0809 20:21:24.374243 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:24.374247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:24.374345 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:24.374486 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:21:24.376430 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.377531 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.379379 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.379998 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.380178 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:24.384375 (+ 4197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:21:24.384379 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:21:24.384383 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:21:24.384411 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:21:24.388279 (+ 3868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:24.388998 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:24.389005 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:24.389010 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:24.389150 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:24.389668 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:24.389673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:24.389676 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:24.390084 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:21:24.390222 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:24.391134 (+ 912us) server_negotiation.cc:299] Negotiation successful
0809 20:21:24.391383 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68,"spinlock_wait_cycles":1792}
I20260809 20:21:24.391930 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:24.375683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41092 (local address 127.5.61.62:41859)
0809 20:21:24.376225 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:24.376232 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:24.376629 (+ 397us) server_negotiation.cc:407] Connection header received
0809 20:21:24.376787 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:24.376790 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:24.376854 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:24.376937 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:21:24.378699 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.379747 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.381658 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.382332 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.382499 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:24.385592 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:21:24.385598 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:21:24.385603 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:21:24.385650 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:21:24.388944 (+ 3294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:24.389672 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:24.389676 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:24.389679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:24.389847 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:24.390222 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:24.390227 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:24.390231 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:24.390479 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:21:24.390653 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:24.391134 (+ 481us) server_negotiation.cc:299] Negotiation successful
0809 20:21:24.391512 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":151,"server-negotiator.queue_time_us":427,"spinlock_wait_cycles":6144,"thread_start_us":305,"threads_started":1}
I20260809 20:21:24.392910 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:24.371508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.3:39605)
0809 20:21:24.372289 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:24.372333 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 20:21:24.374004 (+ 1671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:21:24.374749 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:21:24.374757 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:21:24.375363 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:21:24.376251 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:24.376274 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.377714 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.377717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:24.379231 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:24.379246 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.380263 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.380268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:24.380460 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:24.381180 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:21:24.381202 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:21:24.384181 (+ 2979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:21:24.388427 (+ 4246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:21:24.388434 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:21:24.388445 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:21:24.388866 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:21:24.389262 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:21:24.389267 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:21:24.389270 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:21:24.389526 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:21:24.390224 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:21:24.390238 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:21:24.390598 (+ 360us) client_negotiation.cc:769] Sending connection context
0809 20:21:24.390943 (+ 345us) client_negotiation.cc:240] Negotiation successful
0809 20:21:24.391523 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":279,"threads_started":1}
I20260809 20:21:24.394497 24173 heartbeater.cc:347] Connected to a master server at 127.5.61.62:41859
I20260809 20:21:24.394838 24173 heartbeater.cc:464] Registering TS with master...
I20260809 20:21:24.395658 24173 heartbeater.cc:511] Master 127.5.61.62:41859 requested a full tablet report, sending...
I20260809 20:21:24.397022 23640 ts_manager.cc:194] Registered new tserver with Master: f8eba6e4e390485b998ae5f0585d10d4 (127.5.61.3:40389)
I20260809 20:21:24.397812 23640 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.3@KRBTEST.COM'} at 127.5.61.3:39605
I20260809 20:21:24.406945 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 20:21:24.413400 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:24.407890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41098 (local address 127.5.61.62:41859)
0809 20:21:24.408065 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:24.408069 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:24.408595 (+ 526us) server_negotiation.cc:407] Connection header received
0809 20:21:24.408767 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:24.408771 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:24.408901 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:24.409023 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:21:24.409472 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.410169 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.411633 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.412021 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.412111 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:24.412587 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:21:24.412851 (+ 264us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 20:21:24.413152 (+ 301us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41098: token expired
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 20:21:24.413493 24177 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41098: token expired
W20260809 20:21:24.414139 23551 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.5.61.62:41859)
I20260809 20:21:24.427239 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:24.414604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41104 (local address 127.5.61.62:41859)
0809 20:21:24.414772 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:24.414779 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:24.414801 (+ 22us) server_negotiation.cc:407] Connection header received
0809 20:21:24.415003 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:24.415007 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:24.415059 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:24.415154 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:21:24.416649 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.417325 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.418870 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.419278 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.419412 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:24.421988 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:21:24.421998 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:21:24.422002 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:21:24.422033 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:21:24.424426 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:24.425117 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:24.425124 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:24.425128 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:24.425274 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:24.425743 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:24.425754 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:24.425756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:24.426064 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:21:24.426346 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:24.426658 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 20:21:24.426899 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 20:21:24.430675 23640 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41104:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:21:24.438838 23640 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:21:24.440703 23863 heartbeater.cc:503] Master 127.5.61.62:41859 was elected leader, sending a full tablet report...
I20260809 20:21:24.475879 24190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:24.451355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56356 (local address 127.5.61.2:37737)
0809 20:21:24.452214 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:24.452224 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:24.466686 (+ 14462us) server_negotiation.cc:407] Connection header received
0809 20:21:24.467229 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:24.467255 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:24.467569 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:24.467790 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:21:24.468974 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.470518 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.473496 (+ 2978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.474783 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.474941 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:24.475139 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:24.475361 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 20:21:24.475530 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":366,"threads_started":1}
I20260809 20:21:24.475889 24186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:24.451252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:37737 (local address 127.0.0.1:56356)
0809 20:21:24.466543 (+ 15291us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:24.466568 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 20:21:24.466745 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:21:24.468221 (+ 1476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:21:24.468228 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:21:24.468275 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:21:24.468808 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:24.468820 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.470704 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.470710 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:24.473318 (+ 2608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:24.473335 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.474992 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.474999 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:24.475163 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:24.475197 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 20:21:24.475315 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 20:21:24.475444 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15106,"mutex_wait_us":28,"thread_start_us":18069,"threads_started":1}
I20260809 20:21:24.476619 24191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:24.451761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56720 (local address 127.5.61.3:40389)
0809 20:21:24.462790 (+ 11029us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:24.462804 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:24.465329 (+ 2525us) server_negotiation.cc:407] Connection header received
0809 20:21:24.465448 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:24.465461 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:24.465819 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:24.466075 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:21:24.467329 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.469290 (+ 1961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.472447 (+ 3157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.473990 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.474118 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:24.474385 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:24.475670 (+ 1285us) server_negotiation.cc:299] Negotiation successful
0809 20:21:24.475841 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10796,"thread_start_us":10585,"threads_started":1}
I20260809 20:21:24.476966 24192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:24.451133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:40389 (local address 127.0.0.1:56720)
0809 20:21:24.464389 (+ 13256us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:24.464426 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 20:21:24.464612 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:21:24.466485 (+ 1873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:21:24.466495 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:21:24.466553 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:21:24.467080 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:24.467095 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.469475 (+ 2380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.469480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:24.472097 (+ 2617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:24.472112 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.474611 (+ 2499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.474618 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:24.474833 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:24.474884 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 20:21:24.475032 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 20:21:24.475222 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13020,"mutex_wait_us":1,"spinlock_wait_cycles":4352,"thread_start_us":15724,"threads_started":1}
I20260809 20:21:24.477044 24189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:24.452454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:44147 (local address 127.0.0.1:41148)
0809 20:21:24.469150 (+ 16696us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:24.469167 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:21:24.469301 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:21:24.469875 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:21:24.469880 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:21:24.469905 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:21:24.470356 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:24.470365 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.471814 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.471820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:24.473936 (+ 2116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:24.473950 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.476221 (+ 2271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.476227 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:24.476403 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:24.476439 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 20:21:24.476523 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 20:21:24.476652 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16598,"mutex_wait_us":47,"thread_start_us":11557,"threads_started":1}
I20260809 20:21:24.477885 24193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:24.452669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41148 (local address 127.5.61.1:44147)
0809 20:21:24.467221 (+ 14552us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:24.467232 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:24.469278 (+ 2046us) server_negotiation.cc:407] Connection header received
0809 20:21:24.469485 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:24.469493 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:24.469685 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:24.469883 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:21:24.470519 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.471614 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.474111 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.475638 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.475759 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:24.475981 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:24.477410 (+ 1429us) server_negotiation.cc:299] Negotiation successful
0809 20:21:24.477564 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14358,"thread_start_us":14031,"threads_started":1}
I20260809 20:21:24.480654 24100 tablet_service.cc:1467] Processing CreateTablet for tablet 7df6284bcc16461b9422c4092a0a2bc0 (DEFAULT_TABLE table=test-table [id=686e14918bc14754a26705c511e2f19e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:21:24.482273 24100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7df6284bcc16461b9422c4092a0a2bc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:21:24.482658 23783 tablet_service.cc:1467] Processing CreateTablet for tablet 7df6284bcc16461b9422c4092a0a2bc0 (DEFAULT_TABLE table=test-table [id=686e14918bc14754a26705c511e2f19e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:21:24.484084 23783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7df6284bcc16461b9422c4092a0a2bc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:21:24.480034 23938 tablet_service.cc:1467] Processing CreateTablet for tablet 7df6284bcc16461b9422c4092a0a2bc0 (DEFAULT_TABLE table=test-table [id=686e14918bc14754a26705c511e2f19e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:21:24.484951 23938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7df6284bcc16461b9422c4092a0a2bc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:21:24.497190 24194 tablet_bootstrap.cc:492] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027: Bootstrap starting.
I20260809 20:21:24.498096 24195 tablet_bootstrap.cc:492] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4: Bootstrap starting.
I20260809 20:21:24.501243 24194 tablet_bootstrap.cc:654] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027: Neither blocks nor log segments found. Creating new log.
I20260809 20:21:24.502113 24195 tablet_bootstrap.cc:654] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4: Neither blocks nor log segments found. Creating new log.
I20260809 20:21:24.503042 24194 log.cc:824] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027: Log is configured to *not* fsync() on all Append() calls
I20260809 20:21:24.503333 24195 log.cc:824] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4: Log is configured to *not* fsync() on all Append() calls
I20260809 20:21:24.505371 24196 tablet_bootstrap.cc:492] T 7df6284bcc16461b9422c4092a0a2bc0 P 8ebf53713a3143e581f3e40def807bb5: Bootstrap starting.
I20260809 20:21:24.506632 24195 tablet_bootstrap.cc:492] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4: No bootstrap required, opened a new log
I20260809 20:21:24.506632 24194 tablet_bootstrap.cc:492] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027: No bootstrap required, opened a new log
I20260809 20:21:24.506909 24195 ts_tablet_manager.cc:1397] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 20:21:24.506909 24194 ts_tablet_manager.cc:1397] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260809 20:21:24.508154 24196 tablet_bootstrap.cc:654] T 7df6284bcc16461b9422c4092a0a2bc0 P 8ebf53713a3143e581f3e40def807bb5: Neither blocks nor log segments found. Creating new log.
I20260809 20:21:24.509124 24196 log.cc:824] T 7df6284bcc16461b9422c4092a0a2bc0 P 8ebf53713a3143e581f3e40def807bb5: Log is configured to *not* fsync() on all Append() calls
I20260809 20:21:24.517765 24196 tablet_bootstrap.cc:492] T 7df6284bcc16461b9422c4092a0a2bc0 P 8ebf53713a3143e581f3e40def807bb5: No bootstrap required, opened a new log
I20260809 20:21:24.518029 24196 ts_tablet_manager.cc:1397] T 7df6284bcc16461b9422c4092a0a2bc0 P 8ebf53713a3143e581f3e40def807bb5: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260809 20:21:24.518208 24194 raft_consensus.cc:348] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8eba6e4e390485b998ae5f0585d10d4" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40389 } } peers { permanent_uuid: "17954d03dc0144dfafdfa2d9383e8027" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37737 } } peers { permanent_uuid: "8ebf53713a3143e581f3e40def807bb5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44147 } }
I20260809 20:21:24.518105 24195 raft_consensus.cc:348] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8eba6e4e390485b998ae5f0585d10d4" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40389 } } peers { permanent_uuid: "17954d03dc0144dfafdfa2d9383e8027" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37737 } } peers { permanent_uuid: "8ebf53713a3143e581f3e40def807bb5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44147 } }
I20260809 20:21:24.518615 24194 raft_consensus.cc:374] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:21:24.518726 24194 raft_consensus.cc:729] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17954d03dc0144dfafdfa2d9383e8027, State: Initialized, Role: FOLLOWER
I20260809 20:21:24.518770 24195 raft_consensus.cc:374] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:21:24.518869 24195 raft_consensus.cc:729] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8eba6e4e390485b998ae5f0585d10d4, State: Initialized, Role: FOLLOWER
I20260809 20:21:24.519385 24194 consensus_queue.cc:260] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8eba6e4e390485b998ae5f0585d10d4" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40389 } } peers { permanent_uuid: "17954d03dc0144dfafdfa2d9383e8027" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37737 } } peers { permanent_uuid: "8ebf53713a3143e581f3e40def807bb5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44147 } }
I20260809 20:21:24.519454 24195 consensus_queue.cc:260] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8eba6e4e390485b998ae5f0585d10d4" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40389 } } peers { permanent_uuid: "17954d03dc0144dfafdfa2d9383e8027" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37737 } } peers { permanent_uuid: "8ebf53713a3143e581f3e40def807bb5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44147 } }
I20260809 20:21:24.520576 24173 heartbeater.cc:503] Master 127.5.61.62:41859 was elected leader, sending a full tablet report...
I20260809 20:21:24.521034 24014 heartbeater.cc:503] Master 127.5.61.62:41859 was elected leader, sending a full tablet report...
I20260809 20:21:24.521279 24195 ts_tablet_manager.cc:1428] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260809 20:21:24.522589 24194 ts_tablet_manager.cc:1428] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027: Time spent starting tablet: real 0.016s user 0.010s sys 0.005s
I20260809 20:21:24.529464 24196 raft_consensus.cc:348] T 7df6284bcc16461b9422c4092a0a2bc0 P 8ebf53713a3143e581f3e40def807bb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8eba6e4e390485b998ae5f0585d10d4" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40389 } } peers { permanent_uuid: "17954d03dc0144dfafdfa2d9383e8027" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37737 } } peers { permanent_uuid: "8ebf53713a3143e581f3e40def807bb5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44147 } }
I20260809 20:21:24.529891 24196 raft_consensus.cc:374] T 7df6284bcc16461b9422c4092a0a2bc0 P 8ebf53713a3143e581f3e40def807bb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:21:24.530037 24196 raft_consensus.cc:729] T 7df6284bcc16461b9422c4092a0a2bc0 P 8ebf53713a3143e581f3e40def807bb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebf53713a3143e581f3e40def807bb5, State: Initialized, Role: FOLLOWER
I20260809 20:21:24.530656 24196 consensus_queue.cc:260] T 7df6284bcc16461b9422c4092a0a2bc0 P 8ebf53713a3143e581f3e40def807bb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8eba6e4e390485b998ae5f0585d10d4" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40389 } } peers { permanent_uuid: "17954d03dc0144dfafdfa2d9383e8027" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37737 } } peers { permanent_uuid: "8ebf53713a3143e581f3e40def807bb5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44147 } }
I20260809 20:21:24.532295 24196 ts_tablet_manager.cc:1428] T 7df6284bcc16461b9422c4092a0a2bc0 P 8ebf53713a3143e581f3e40def807bb5: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260809 20:21:24.620087 24174 tablet.cc:2369] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 20:21:24.652354 24016 tablet.cc:2369] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 20:21:24.654098 23864 tablet.cc:2369] T 7df6284bcc16461b9422c4092a0a2bc0 P 8ebf53713a3143e581f3e40def807bb5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:21:24.761499 24203 raft_consensus.cc:482] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:21:24.761842 24203 raft_consensus.cc:504] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8eba6e4e390485b998ae5f0585d10d4" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40389 } } peers { permanent_uuid: "17954d03dc0144dfafdfa2d9383e8027" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37737 } } peers { permanent_uuid: "8ebf53713a3143e581f3e40def807bb5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44147 } }
I20260809 20:21:24.763166 24202 raft_consensus.cc:482] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:21:24.763433 24202 raft_consensus.cc:504] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8eba6e4e390485b998ae5f0585d10d4" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40389 } } peers { permanent_uuid: "17954d03dc0144dfafdfa2d9383e8027" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37737 } } peers { permanent_uuid: "8ebf53713a3143e581f3e40def807bb5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44147 } }
I20260809 20:21:24.763983 24203 leader_election.cc:288] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8eba6e4e390485b998ae5f0585d10d4 (127.5.61.3:40389), 8ebf53713a3143e581f3e40def807bb5 (127.5.61.1:44147)
I20260809 20:21:24.771811 24202 leader_election.cc:288] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17954d03dc0144dfafdfa2d9383e8027 (127.5.61.2:37737), 8ebf53713a3143e581f3e40def807bb5 (127.5.61.1:44147)
I20260809 20:21:24.775954 24193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:24.764666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:39235 (local address 127.5.61.1:44147)
0809 20:21:24.764865 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:24.764871 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:24.765968 (+ 1097us) server_negotiation.cc:407] Connection header received
0809 20:21:24.766271 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:24.766277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:24.766369 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:24.766436 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:21:24.767509 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.769036 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.772392 (+ 3356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.774233 (+ 1841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.774384 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:24.774533 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:24.775357 (+ 824us) server_negotiation.cc:299] Negotiation successful
0809 20:21:24.775580 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 20:21:24.776995 24190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:24.767142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:56343 (local address 127.5.61.2:37737)
0809 20:21:24.767390 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:24.767397 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:24.767420 (+ 23us) server_negotiation.cc:407] Connection header received
0809 20:21:24.767486 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:24.767493 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:24.767600 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:24.767703 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:21:24.768380 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.769791 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.773092 (+ 3301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.774808 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.774984 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:24.775119 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:24.776420 (+ 1301us) server_negotiation.cc:299] Negotiation successful
0809 20:21:24.776606 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 20:21:24.777004 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:24.764444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:44147 (local address 127.5.61.2:39235)
0809 20:21:24.765858 (+ 1414us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:24.765876 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 20:21:24.766083 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:21:24.766674 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:21:24.766680 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:21:24.766713 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:21:24.767275 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:24.767292 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.769257 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.769265 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:24.772147 (+ 2882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:24.772184 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.774854 (+ 2670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.774864 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:24.775116 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:24.775169 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 20:21:24.776285 (+ 1116us) client_negotiation.cc:240] Negotiation successful
0809 20:21:24.776474 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1269,"thread_start_us":653,"threads_started":1}
I20260809 20:21:24.778261 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:24.765860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:37737 (local address 127.5.61.3:56343)
0809 20:21:24.766291 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:24.766309 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 20:21:24.766397 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:21:24.767898 (+ 1501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:21:24.767905 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:21:24.767939 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:21:24.768180 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:24.768194 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.770056 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.770064 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:24.772741 (+ 2677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:24.772762 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.775261 (+ 2499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.775270 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:24.775593 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:24.775646 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 20:21:24.775752 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 20:21:24.775937 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286}
I20260809 20:21:24.779287 23963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7df6284bcc16461b9422c4092a0a2bc0" candidate_uuid: "f8eba6e4e390485b998ae5f0585d10d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17954d03dc0144dfafdfa2d9383e8027" is_pre_election: true
I20260809 20:21:24.779834 23963 raft_consensus.cc:2454] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8eba6e4e390485b998ae5f0585d10d4 in term 0.
I20260809 20:21:24.779763 23807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7df6284bcc16461b9422c4092a0a2bc0" candidate_uuid: "17954d03dc0144dfafdfa2d9383e8027" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebf53713a3143e581f3e40def807bb5" is_pre_election: true
I20260809 20:21:24.780347 23807 raft_consensus.cc:2454] T 7df6284bcc16461b9422c4092a0a2bc0 P 8ebf53713a3143e581f3e40def807bb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17954d03dc0144dfafdfa2d9383e8027 in term 0.
I20260809 20:21:24.780797 24043 leader_election.cc:302] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17954d03dc0144dfafdfa2d9383e8027, f8eba6e4e390485b998ae5f0585d10d4; no voters:
I20260809 20:21:24.781035 23889 leader_election.cc:302] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17954d03dc0144dfafdfa2d9383e8027, 8ebf53713a3143e581f3e40def807bb5; no voters:
I20260809 20:21:24.781358 24202 raft_consensus.cc:2781] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:21:24.781405 24203 raft_consensus.cc:2781] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:21:24.781527 24202 raft_consensus.cc:482] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:21:24.781538 24203 raft_consensus.cc:482] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:21:24.781638 24203 raft_consensus.cc:3037] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:21:24.781646 24202 raft_consensus.cc:3037] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:21:24.783550 24191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:24.764193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:43505 (local address 127.5.61.3:40389)
0809 20:21:24.764414 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:24.764423 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:24.765662 (+ 1239us) server_negotiation.cc:407] Connection header received
0809 20:21:24.765974 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:24.765979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:24.766072 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:24.766138 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:21:24.777652 (+ 11514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.778779 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.781660 (+ 2881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.782569 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.782649 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:24.782713 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:24.783177 (+ 464us) server_negotiation.cc:299] Negotiation successful
0809 20:21:24.783298 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 20:21:24.783661 24208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:24.764152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:40389 (local address 127.5.61.2:43505)
0809 20:21:24.765541 (+ 1389us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:24.765566 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 20:21:24.765795 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:21:24.766400 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:21:24.766408 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:21:24.766447 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:21:24.777506 (+ 11059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:24.777521 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.778981 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.778989 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:24.781420 (+ 2431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:24.781432 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.782891 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.782897 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:24.783012 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:24.783033 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 20:21:24.783150 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 20:21:24.783281 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1205,"thread_start_us":2580,"threads_started":1}
I20260809 20:21:24.784442 24124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7df6284bcc16461b9422c4092a0a2bc0" candidate_uuid: "17954d03dc0144dfafdfa2d9383e8027" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8eba6e4e390485b998ae5f0585d10d4" is_pre_election: true
I20260809 20:21:24.785013 24203 raft_consensus.cc:504] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8eba6e4e390485b998ae5f0585d10d4" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40389 } } peers { permanent_uuid: "17954d03dc0144dfafdfa2d9383e8027" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37737 } } peers { permanent_uuid: "8ebf53713a3143e581f3e40def807bb5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44147 } }
I20260809 20:21:24.785550 24202 raft_consensus.cc:504] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8eba6e4e390485b998ae5f0585d10d4" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40389 } } peers { permanent_uuid: "17954d03dc0144dfafdfa2d9383e8027" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37737 } } peers { permanent_uuid: "8ebf53713a3143e581f3e40def807bb5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44147 } }
I20260809 20:21:24.786154 24124 raft_consensus.cc:2379] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 17954d03dc0144dfafdfa2d9383e8027 in current term 1: Already voted for candidate f8eba6e4e390485b998ae5f0585d10d4 in this term.
I20260809 20:21:24.786119 24203 leader_election.cc:288] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027 [CANDIDATE]: Term 1 election: Requested vote from peers f8eba6e4e390485b998ae5f0585d10d4 (127.5.61.3:40389), 8ebf53713a3143e581f3e40def807bb5 (127.5.61.1:44147)
I20260809 20:21:24.786214 23807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7df6284bcc16461b9422c4092a0a2bc0" candidate_uuid: "17954d03dc0144dfafdfa2d9383e8027" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebf53713a3143e581f3e40def807bb5"
I20260809 20:21:24.786221 24123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7df6284bcc16461b9422c4092a0a2bc0" candidate_uuid: "17954d03dc0144dfafdfa2d9383e8027" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8eba6e4e390485b998ae5f0585d10d4"
I20260809 20:21:24.786413 23807 raft_consensus.cc:3037] T 7df6284bcc16461b9422c4092a0a2bc0 P 8ebf53713a3143e581f3e40def807bb5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:21:24.787276 24202 leader_election.cc:288] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4 [CANDIDATE]: Term 1 election: Requested vote from peers 17954d03dc0144dfafdfa2d9383e8027 (127.5.61.2:37737), 8ebf53713a3143e581f3e40def807bb5 (127.5.61.1:44147)
I20260809 20:21:24.787703 23963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7df6284bcc16461b9422c4092a0a2bc0" candidate_uuid: "f8eba6e4e390485b998ae5f0585d10d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17954d03dc0144dfafdfa2d9383e8027"
I20260809 20:21:24.787984 23963 raft_consensus.cc:2379] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f8eba6e4e390485b998ae5f0585d10d4 in current term 1: Already voted for candidate 17954d03dc0144dfafdfa2d9383e8027 in this term.
I20260809 20:21:24.789796 23807 raft_consensus.cc:2454] T 7df6284bcc16461b9422c4092a0a2bc0 P 8ebf53713a3143e581f3e40def807bb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17954d03dc0144dfafdfa2d9383e8027 in term 1.
I20260809 20:21:24.790105 24210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:24.765854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:44147 (local address 127.5.61.3:44397)
0809 20:21:24.777546 (+ 11692us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:24.777565 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 20:21:24.777756 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:21:24.778497 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:21:24.778503 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:21:24.778534 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:21:24.778962 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:24.778974 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.780568 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.780574 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:24.787750 (+ 7176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:21:24.787767 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.789457 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.789462 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:21:24.789580 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:24.789603 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 20:21:24.789678 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 20:21:24.789762 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11561,"mutex_wait_us":619,"thread_start_us":10696,"threads_started":1}
I20260809 20:21:24.790167 24193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:24.766135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:44397 (local address 127.5.61.1:44147)
0809 20:21:24.776234 (+ 10099us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:24.776239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:24.777634 (+ 1395us) server_negotiation.cc:407] Connection header received
0809 20:21:24.777930 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:24.777941 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:24.778054 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:24.778120 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:21:24.779182 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.780356 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.787944 (+ 7588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.789280 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.789414 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:24.789528 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:24.789734 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 20:21:24.789872 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9975,"thread_start_us":11672,"threads_started":1}
I20260809 20:21:24.790196 23889 leader_election.cc:302] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17954d03dc0144dfafdfa2d9383e8027, 8ebf53713a3143e581f3e40def807bb5; no voters: f8eba6e4e390485b998ae5f0585d10d4
I20260809 20:21:24.790537 24203 raft_consensus.cc:2781] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:21:24.790789 23807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7df6284bcc16461b9422c4092a0a2bc0" candidate_uuid: "f8eba6e4e390485b998ae5f0585d10d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebf53713a3143e581f3e40def807bb5" is_pre_election: true
I20260809 20:21:24.791069 24203 raft_consensus.cc:686] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027 [term 1 LEADER]: Becoming Leader. State: Replica: 17954d03dc0144dfafdfa2d9383e8027, State: Running, Role: LEADER
I20260809 20:21:24.790956 23806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7df6284bcc16461b9422c4092a0a2bc0" candidate_uuid: "f8eba6e4e390485b998ae5f0585d10d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebf53713a3143e581f3e40def807bb5"
I20260809 20:21:24.791296 23807 raft_consensus.cc:2379] T 7df6284bcc16461b9422c4092a0a2bc0 P 8ebf53713a3143e581f3e40def807bb5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f8eba6e4e390485b998ae5f0585d10d4 in current term 1: Already voted for candidate 17954d03dc0144dfafdfa2d9383e8027 in this term.
I20260809 20:21:24.791628 24203 consensus_queue.cc:237] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8eba6e4e390485b998ae5f0585d10d4" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40389 } } peers { permanent_uuid: "17954d03dc0144dfafdfa2d9383e8027" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37737 } } peers { permanent_uuid: "8ebf53713a3143e581f3e40def807bb5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44147 } }
I20260809 20:21:24.791843 24045 leader_election.cc:302] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f8eba6e4e390485b998ae5f0585d10d4; no voters: 17954d03dc0144dfafdfa2d9383e8027, 8ebf53713a3143e581f3e40def807bb5
I20260809 20:21:24.792171 24202 raft_consensus.cc:2726] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 20:21:24.796044 23640 catalog_manager.cc:5495] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 17954d03dc0144dfafdfa2d9383e8027 (127.5.61.2). New cstate: current_term: 1 leader_uuid: "17954d03dc0144dfafdfa2d9383e8027" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8eba6e4e390485b998ae5f0585d10d4" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17954d03dc0144dfafdfa2d9383e8027" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 37737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebf53713a3143e581f3e40def807bb5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44147 } health_report { overall_health: UNKNOWN } } }
I20260809 20:21:24.840018 24190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:24.833653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56366 (local address 127.5.61.2:37737)
0809 20:21:24.833872 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:24.833879 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:24.833999 (+ 120us) server_negotiation.cc:407] Connection header received
0809 20:21:24.834341 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:24.834345 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:24.834431 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:24.834676 (+ 245us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:21:24.835083 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.836101 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.837579 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.838049 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.838197 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:24.838801 (+ 604us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:21:24.839309 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:21:24.839480 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:24.839595 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 20:21:24.839694 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 20:21:24.848459 24124 raft_consensus.cc:1261] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4 [term 1 FOLLOWER]: Refusing update from remote peer 17954d03dc0144dfafdfa2d9383e8027: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:21:24.848438 23807 raft_consensus.cc:1261] T 7df6284bcc16461b9422c4092a0a2bc0 P 8ebf53713a3143e581f3e40def807bb5 [term 1 FOLLOWER]: Refusing update from remote peer 17954d03dc0144dfafdfa2d9383e8027: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:21:24.849366 24212 consensus_queue.cc:1035] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebf53713a3143e581f3e40def807bb5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 44147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:21:24.849735 24203 consensus_queue.cc:1035] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8eba6e4e390485b998ae5f0585d10d4" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 40389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:21:24.863281 24219 mvcc.cc:205] Tried to move back new op lower bound from 7316713000328794112 to 7316713000110837760. Current Snapshot: MvccSnapshot[applied={T|T < 7316713000328794112}]
I20260809 20:21:24.874812 23640 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:21:24.875070 23640 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:21:24.878688 23640 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906: Sending DeleteTablet for 3 replicas of tablet 7df6284bcc16461b9422c4092a0a2bc0
I20260809 20:21:24.879789 24100 tablet_service.cc:1514] Processing DeleteTablet for tablet 7df6284bcc16461b9422c4092a0a2bc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:21:24 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:56720
I20260809 20:21:24.880414 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 23684
I20260809 20:21:24.880537 23783 tablet_service.cc:1514] Processing DeleteTablet for tablet 7df6284bcc16461b9422c4092a0a2bc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:21:24 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:41148
I20260809 20:21:24.880798 24227 tablet_replica.cc:330] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4: stopping tablet replica
I20260809 20:21:24.881253 24227 raft_consensus.cc:2229] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:21:24.881249 23938 tablet_service.cc:1514] Processing DeleteTablet for tablet 7df6284bcc16461b9422c4092a0a2bc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:21:24 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:56356
I20260809 20:21:24.881551 24228 tablet_replica.cc:330] T 7df6284bcc16461b9422c4092a0a2bc0 P 8ebf53713a3143e581f3e40def807bb5: stopping tablet replica
I20260809 20:21:24.881732 24227 raft_consensus.cc:2258] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:21:24.882050 24228 raft_consensus.cc:2229] T 7df6284bcc16461b9422c4092a0a2bc0 P 8ebf53713a3143e581f3e40def807bb5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:21:24.882591 24228 raft_consensus.cc:2258] T 7df6284bcc16461b9422c4092a0a2bc0 P 8ebf53713a3143e581f3e40def807bb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:21:24.883786 24227 ts_tablet_manager.cc:1905] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:21:24.884656 24228 ts_tablet_manager.cc:1905] T 7df6284bcc16461b9422c4092a0a2bc0 P 8ebf53713a3143e581f3e40def807bb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:21:24.890693 24227 ts_tablet_manager.cc:1918] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:21:24.890852 24227 log.cc:1196] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/7df6284bcc16461b9422c4092a0a2bc0
I20260809 20:21:24.891345 24227 ts_tablet_manager.cc:1939] T 7df6284bcc16461b9422c4092a0a2bc0 P f8eba6e4e390485b998ae5f0585d10d4: Deleting consensus metadata
I20260809 20:21:24.891999 24228 ts_tablet_manager.cc:1918] T 7df6284bcc16461b9422c4092a0a2bc0 P 8ebf53713a3143e581f3e40def807bb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:21:24.892235 24228 log.cc:1196] T 7df6284bcc16461b9422c4092a0a2bc0 P 8ebf53713a3143e581f3e40def807bb5: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/7df6284bcc16461b9422c4092a0a2bc0
I20260809 20:21:24.892899 24228 ts_tablet_manager.cc:1939] T 7df6284bcc16461b9422c4092a0a2bc0 P 8ebf53713a3143e581f3e40def807bb5: Deleting consensus metadata
I20260809 20:21:24.893216 23621 catalog_manager.cc:4841] TS f8eba6e4e390485b998ae5f0585d10d4 (127.5.61.3:40389): tablet 7df6284bcc16461b9422c4092a0a2bc0 (table test-table [id=686e14918bc14754a26705c511e2f19e]) successfully deleted
I20260809 20:21:24.894166 24229 tablet_replica.cc:330] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027: stopping tablet replica
I20260809 20:21:24.895005 23623 catalog_manager.cc:4841] TS 8ebf53713a3143e581f3e40def807bb5 (127.5.61.1:44147): tablet 7df6284bcc16461b9422c4092a0a2bc0 (table test-table [id=686e14918bc14754a26705c511e2f19e]) successfully deleted
I20260809 20:21:24.895036 24229 raft_consensus.cc:2229] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027 [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:21:24.896266 24229 raft_consensus.cc:2258] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:21:24.898262 24229 ts_tablet_manager.cc:1905] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:21:24.901614 24193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:24.882045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41164 (local address 127.5.61.1:44147)
0809 20:21:24.882228 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:24.882234 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:24.882255 (+ 21us) server_negotiation.cc:407] Connection header received
0809 20:21:24.882328 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:24.882336 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:24.882432 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:24.882554 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:21:24.885089 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.886646 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.888917 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:24.889570 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:24.889709 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:24.893718 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:21:24.893728 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:21:24.893743 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:21:24.893781 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:21:24.897294 (+ 3513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:24.898448 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:24.898460 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:24.898463 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:24.898633 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:24.899119 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:24.899124 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:24.899128 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:24.899588 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:21:24.900904 (+ 1316us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:24.901054 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 20:21:24.901265 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":57}
I20260809 20:21:24.902529 23852 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:21:24.906550 24229 ts_tablet_manager.cc:1918] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:21:24.906720 24229 log.cc:1196] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/7df6284bcc16461b9422c4092a0a2bc0
I20260809 20:21:24.907255 24229 ts_tablet_manager.cc:1939] T 7df6284bcc16461b9422c4092a0a2bc0 P 17954d03dc0144dfafdfa2d9383e8027: Deleting consensus metadata
I20260809 20:21:24.908665 23620 catalog_manager.cc:4841] TS 17954d03dc0144dfafdfa2d9383e8027 (127.5.61.2:37737): tablet 7df6284bcc16461b9422c4092a0a2bc0 (table test-table [id=686e14918bc14754a26705c511e2f19e]) successfully deleted
I20260809 20:21:25.046658 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 23684
I20260809 20:21:25.074756 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 23867
I20260809 20:21:25.093703 24190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:25.075629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56382 (local address 127.5.61.2:37737)
0809 20:21:25.075815 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:25.075821 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:25.075872 (+ 51us) server_negotiation.cc:407] Connection header received
0809 20:21:25.076102 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:25.076107 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:25.076203 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:25.076318 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:21:25.077734 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:25.078468 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:25.079922 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:25.080503 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:25.080649 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:25.087424 (+ 6775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:21:25.087442 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:21:25.087457 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:21:25.087504 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:21:25.090641 (+ 3137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:25.091393 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:25.091400 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:25.091403 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:25.091613 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:25.092058 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:25.092063 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:25.092067 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:25.092465 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:21:25.092777 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:25.093064 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 20:21:25.093336 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 20:21:25.094635 24007 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:21:25.206561 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 23867
I20260809 20:21:25.228688 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 24021
I20260809 20:21:25.242597 24191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:25.229395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56730 (local address 127.5.61.3:40389)
0809 20:21:25.229578 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 20:21:25.229584 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:21:25.229656 (+ 72us) server_negotiation.cc:407] Connection header received
0809 20:21:25.229887 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:21:25.229899 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:21:25.229972 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:21:25.230081 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:21:25.231357 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:25.232147 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:25.233486 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:21:25.233883 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:21:25.234025 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:21:25.236790 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:21:25.236804 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:21:25.236820 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:21:25.236852 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:21:25.239749 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:25.240441 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:25.240449 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:25.240452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:25.240589 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:21:25.240999 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:21:25.241004 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:21:25.241007 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:21:25.241358 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:21:25.241652 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0809 20:21:25.242002 (+ 350us) server_negotiation.cc:299] Negotiation successful
0809 20:21:25.242222 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 20:21:25.243237 24168 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:21:25.334267 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 24021
I20260809 20:21:25.356391 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 23598
I20260809 20:21:25.357534 23674 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:21:25.471724 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 23598
I20260809 20:21:45.497076 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:41859
--webserver_interface=127.5.61.62
--webserver_port=32847
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:41859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:21:45.834164 24234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:21:45.834529 24234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:21:45.834717 24234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:21:45.841989 24234 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 20:21:45.842124 24234 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 20:21:45.842218 24234 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 20:21:45.842315 24234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:21:45.842415 24234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:21:45.842535 24234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:21:45.842623 24234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:21:45.842720 24234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:21:45.852468 24234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:41859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:41859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=32847
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:21:45.853725 24234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:21:45.855046 24234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:21:45.866989 24239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:21:45.867027 24240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:21:45.867484 24242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:21:45.867810 24234 server_base.cc:1034] running on GCE node
I20260809 20:21:45.868467 24234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:21:45.869772 24234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:21:45.871044 24234 hybrid_clock.cc:648] HybridClock initialized: now 1786306905870959 us; error 51 us; skew 500 ppm
I20260809 20:21:45.871356 24234 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:21:45 dist-test-slave-6k22 krb5kdc[22955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306905, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:21:45.880656 24234 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:21:45.882215 24234 webserver.cc:460] Webserver started at http://127.5.61.62:32847/ using document root <none> and password file <none>
I20260809 20:21:45.882870 24234 fs_manager.cc:359] Metadata directory not provided
I20260809 20:21:45.882956 24234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:21:45.888296 24234 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 20:21:45.891319 24250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:45.892650 24234 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 20:21:45.892814 24234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "8ef1eae90ca245239d5944f2d0e0d906"
format_stamp: "Formatted at 2026-08-09 20:20:59 on dist-test-slave-6k22"
server_key: "cb7a94fcb2a7d1a27eb7c976fca49b0e"
server_key_iv: "b069ff8c22cc92a1c33d495ff4cca014"
server_key_version: "encryptionkey@0"
I20260809 20:21:45.893774 24234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:21:45.908416 24234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:21:45.912760 24234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:21:45.913048 24234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:21:45.930389 24234 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:41859
I20260809 20:21:45.930435 24309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:41859 every 8 connection(s)
I20260809 20:21:45.931866 24234 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:21:45.937201 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 24234
I20260809 20:21:45.938249 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:44147
--local_ip_for_outbound_sockets=127.5.61.1
--tserver_master_addrs=127.5.61.62:41859
--webserver_port=40405
--webserver_interface=127.5.61.1
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 20:21:45.939397 24312 sys_catalog.cc:263] Verifying existing consensus state
I20260809 20:21:45.944415 24312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906: Bootstrap starting.
I20260809 20:21:45.966099 24312 log.cc:824] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906: Log is configured to *not* fsync() on all Append() calls
I20260809 20:21:45.982550 24312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 20:21:45.983110 24312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906: Bootstrap complete.
I20260809 20:21:45.992261 24312 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef1eae90ca245239d5944f2d0e0d906" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41859 } }
I20260809 20:21:45.993147 24312 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef1eae90ca245239d5944f2d0e0d906, State: Initialized, Role: FOLLOWER
I20260809 20:21:45.993844 24312 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef1eae90ca245239d5944f2d0e0d906" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41859 } }
I20260809 20:21:45.994110 24312 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:21:45.994246 24312 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:21:45.994436 24312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 2 FOLLOWER]: Advancing to term 3
I20260809 20:21:45.998492 24312 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef1eae90ca245239d5944f2d0e0d906" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41859 } }
I20260809 20:21:45.998977 24312 leader_election.cc:302] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ef1eae90ca245239d5944f2d0e0d906; no voters:
I20260809 20:21:45.999704 24312 leader_election.cc:288] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 20:21:45.999830 24316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 20:21:46.002142 24312 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:21:46.003851 24316 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [term 3 LEADER]: Becoming Leader. State: Replica: 8ef1eae90ca245239d5944f2d0e0d906, State: Running, Role: LEADER
I20260809 20:21:46.004949 24316 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef1eae90ca245239d5944f2d0e0d906" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41859 } }
I20260809 20:21:46.008102 24317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8ef1eae90ca245239d5944f2d0e0d906" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef1eae90ca245239d5944f2d0e0d906" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41859 } } }
I20260809 20:21:46.008430 24317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [sys.catalog]: This master's current role is: LEADER
I20260809 20:21:46.007788 24318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ef1eae90ca245239d5944f2d0e0d906. Latest consensus state: current_term: 3 leader_uuid: "8ef1eae90ca245239d5944f2d0e0d906" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef1eae90ca245239d5944f2d0e0d906" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 41859 } } }
I20260809 20:21:46.009107 24318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906 [sys.catalog]: This master's current role is: LEADER
I20260809 20:21:46.016841 24326 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:21:46.027424 24326 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:21:46.029320 24326 catalog_manager.cc:1259] Loaded cluster ID: 67627d0ed1f84966b661aed141f857a7
I20260809 20:21:46.029522 24326 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:21:46.034853 24326 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:21:46.037539 24326 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906: Loaded TSK: 0
I20260809 20:21:46.037810 24326 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8ef1eae90ca245239d5944f2d0e0d906: Loaded TSK: 1
F20260809 20:21:46.039357 24326 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f840d8449ad google::LogMessage::Fail() at ??:0
@ 0x7f840d84788a google::LogMessage::SendToLog() at ??:0
@ 0x7f840d8444de google::LogMessage::Flush() at ??:0
@ 0x7f840d845a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f84115a87d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
I20260809 20:21:46.059930 24334 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f841159773e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f84232771c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f8423276783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f84233022a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f84233020c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f842334c5e9 std::function<>::operator()() at ??:0
@ 0x7f842327acdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f8423278c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f8423300610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f8423300439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f8410490345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f841049391c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f8410493799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f841046eeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f84137486db start_thread at ??:0
@ 0x7f840c6de61f clone at ??:0
W20260809 20:21:46.323355 24314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:21:46.323707 24314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:21:46.323825 24314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:21:46.331169 24314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:21:46.331267 24314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:21:46.331475 24314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:21:46.341122 24314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:44147
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=40405
--tserver_master_addrs=127.5.61.62:41859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:21:46.342444 24314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:21:46.343804 24314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:21:46.357280 24367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:21:46.357278 24365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:21:46.357599 24364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:21:46.358229 24314 server_base.cc:1034] running on GCE node
I20260809 20:21:46.358862 24314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:21:46.359836 24314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:21:46.361068 24314 hybrid_clock.cc:648] HybridClock initialized: now 1786306906361023 us; error 59 us; skew 500 ppm
I20260809 20:21:46.361415 24314 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:21:46 dist-test-slave-6k22 krb5kdc[22955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306906, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:21:46.364599 24314 init.cc:377] Logged in from keytab as kudu/127.5.61.1@KRBTEST.COM (short username kudu)
I20260809 20:21:46.365998 24314 webserver.cc:460] Webserver started at http://127.5.61.1:40405/ using document root <none> and password file <none>
I20260809 20:21:46.366678 24314 fs_manager.cc:359] Metadata directory not provided
I20260809 20:21:46.366845 24314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:21:46.372043 24314 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:21:46.374881 24378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:46.376183 24314 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:21:46.376354 24314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "8ebf53713a3143e581f3e40def807bb5"
format_stamp: "Formatted at 2026-08-09 20:21:00 on dist-test-slave-6k22"
server_key: "5f4f6ce7ff00d0d10209fa47881a28f6"
server_key_iv: "b2ffe81e908d47d898b5f33299bd776f"
server_key_version: "encryptionkey@0"
I20260809 20:21:46.377359 24314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:21:46.390864 24314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:21:46.395355 24314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:21:46.395680 24314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:21:46.396615 24314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:21:46.397945 24314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:21:46.398069 24314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:46.398202 24314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:21:46.398298 24314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:46.434393 24314 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:44147
I20260809 20:21:46.434445 24528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:44147 every 8 connection(s)
I20260809 20:21:46.435947 24314 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:21:46.440897 24531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:46.438983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.1:45757)
0809 20:21:46.439827 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:46.440023 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":315,"threads_started":1}
W20260809 20:21:46.441557 24529 heartbeater.cc:644] Failed to heartbeat to 127.5.61.62:41859 (0 consecutive failures): Network error: Failed to ping master at 127.5.61.62:41859: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
I20260809 20:21:46.442301 24531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:46.442016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.1:40613)
0809 20:21:46.442141 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:46.442205 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 20:21:46.443092 24531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:46.442790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.1:37965)
0809 20:21:46.442908 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:46.442974 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 20:21:46.443446 24529 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 20:21:46.443975 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 24314
I20260809 20:21:46.444706 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:37737
--local_ip_for_outbound_sockets=127.5.61.2
--tserver_master_addrs=127.5.61.62:41859
--webserver_port=42803
--webserver_interface=127.5.61.2
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:21:46.777264 24532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:21:46.777527 24532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:21:46.777621 24532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:21:46.785288 24532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:21:46.785389 24532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:21:46.785496 24532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:21:46.794876 24532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:37737
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=42803
--tserver_master_addrs=127.5.61.62:41859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:21:46.796080 24532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:21:46.797348 24532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:21:46.810336 24537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:21:46.810336 24538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:21:46.810400 24541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:21:46.811286 24532 server_base.cc:1034] running on GCE node
I20260809 20:21:46.812023 24532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:21:46.813001 24532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:21:46.814195 24532 hybrid_clock.cc:648] HybridClock initialized: now 1786306906814139 us; error 45 us; skew 500 ppm
I20260809 20:21:46.814508 24532 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:21:46 dist-test-slave-6k22 krb5kdc[22955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306906, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:21:46.817562 24532 init.cc:377] Logged in from keytab as kudu/127.5.61.2@KRBTEST.COM (short username kudu)
I20260809 20:21:46.818899 24532 webserver.cc:460] Webserver started at http://127.5.61.2:42803/ using document root <none> and password file <none>
I20260809 20:21:46.819499 24532 fs_manager.cc:359] Metadata directory not provided
I20260809 20:21:46.819665 24532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:21:46.824704 24532 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 20:21:46.827286 24550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:46.828617 24532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 20:21:46.828802 24532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "17954d03dc0144dfafdfa2d9383e8027"
format_stamp: "Formatted at 2026-08-09 20:21:01 on dist-test-slave-6k22"
server_key: "51d27605186ccefec7fe380c09ea5db5"
server_key_iv: "c28426eb1bab84ba6634008194d22c9d"
server_key_version: "encryptionkey@0"
I20260809 20:21:46.829752 24532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:21:46.846458 24532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:21:46.850191 24532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:21:46.850471 24532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:21:46.851432 24532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:21:46.852682 24532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:21:46.852843 24532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:46.853001 24532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:21:46.853106 24532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:46.886605 24532 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:37737
I20260809 20:21:46.886670 24686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:37737 every 8 connection(s)
I20260809 20:21:46.888221 24532 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:21:46.892719 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 24532
I20260809 20:21:46.892992 24689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:46.891030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.2:50257)
0809 20:21:46.891849 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:46.892059 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":287,"threads_started":1}
I20260809 20:21:46.893456 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:40389
--local_ip_for_outbound_sockets=127.5.61.3
--tserver_master_addrs=127.5.61.62:41859
--webserver_port=38339
--webserver_interface=127.5.61.3
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:21:46.893815 24687 heartbeater.cc:644] Failed to heartbeat to 127.5.61.62:41859 (0 consecutive failures): Network error: Failed to ping master at 127.5.61.62:41859: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
I20260809 20:21:46.894632 24689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:46.894298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.2:40681)
0809 20:21:46.894445 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:46.894524 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 20:21:46.895597 24689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:46.895138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.2:48041)
0809 20:21:46.895322 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:46.895422 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 20:21:46.895885 24687 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 20:21:47.225442 24690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:21:47.225764 24690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:21:47.225940 24690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:21:47.233093 24690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:21:47.233234 24690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:21:47.233402 24690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:21:47.242856 24690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:40389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=38339
--tserver_master_addrs=127.5.61.62:41859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:21:47.244174 24690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:21:47.245440 24690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:21:47.258791 24698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:21:47.258884 24696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:21:47.258857 24700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:21:47.259877 24690 server_base.cc:1034] running on GCE node
I20260809 20:21:47.260488 24690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:21:47.261446 24690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:21:47.262686 24690 hybrid_clock.cc:648] HybridClock initialized: now 1786306907262617 us; error 55 us; skew 500 ppm
I20260809 20:21:47.262964 24690 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:21:47 dist-test-slave-6k22 krb5kdc[22955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306907, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:21:47.266048 24690 init.cc:377] Logged in from keytab as kudu/127.5.61.3@KRBTEST.COM (short username kudu)
I20260809 20:21:47.267534 24690 webserver.cc:460] Webserver started at http://127.5.61.3:38339/ using document root <none> and password file <none>
I20260809 20:21:47.268296 24690 fs_manager.cc:359] Metadata directory not provided
I20260809 20:21:47.268451 24690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:21:47.273599 24690 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 20:21:47.276283 24709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:47.277561 24690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:21:47.277735 24690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "f8eba6e4e390485b998ae5f0585d10d4"
format_stamp: "Formatted at 2026-08-09 20:21:01 on dist-test-slave-6k22"
server_key: "f50ff5258ed56bab4e582f3a940683db"
server_key_iv: "3859d0f4618d69f85b9b79f6ffc74f60"
server_key_version: "encryptionkey@0"
I20260809 20:21:47.278728 24690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:21:47.300683 24690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:21:47.304486 24690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:21:47.304756 24690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:21:47.305697 24690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:21:47.306985 24690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:21:47.307111 24690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:47.307240 24690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:21:47.307327 24690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:21:47.342373 24690 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:40389
I20260809 20:21:47.342458 24846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:40389 every 8 connection(s)
I20260809 20:21:47.343894 24690 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:21:47.348692 24849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:47.346813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.3:42421)
0809 20:21:47.347543 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:47.347822 (+ 279us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":282,"threads_started":1}
W20260809 20:21:47.349300 24847 heartbeater.cc:644] Failed to heartbeat to 127.5.61.62:41859 (0 consecutive failures): Network error: Failed to ping master at 127.5.61.62:41859: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
I20260809 20:21:47.350049 24849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:47.349754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.3:57101)
0809 20:21:47.349890 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:47.349955 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 20:21:47.350847 24849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:47.350540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.3:42627)
0809 20:21:47.350674 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:47.350749 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 20:21:47.351125 24847 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 20:21:47.351450 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 24690
I20260809 20:21:47.444705 24851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:47.443964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.1:46793)
0809 20:21:47.444398 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:47.444482 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":207,"threads_started":1}
I20260809 20:21:47.897095 24852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:47.896342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.2:40493)
0809 20:21:47.896784 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:47.896877 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":220,"threads_started":1}
I20260809 20:21:48.352617 24853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:48.351741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.3:34699)
0809 20:21:48.352247 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:48.352372 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":273,"threads_started":1}
I20260809 20:21:48.446502 24854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:48.445567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.1:44459)
0809 20:21:48.446171 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:48.446264 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":340,"threads_started":1}
I20260809 20:21:48.898620 24855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:48.897888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.2:42659)
0809 20:21:48.898357 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:48.898458 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":242,"threads_started":1}
I20260809 20:21:49.354210 24856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:49.353454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.3:41213)
0809 20:21:49.353961 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:49.354037 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":278,"threads_started":1}
I20260809 20:21:49.448172 24858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:49.447478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.1:41221)
0809 20:21:49.447943 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:49.448016 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":272,"threads_started":1}
I20260809 20:21:49.900161 24859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:49.899432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.2:40093)
0809 20:21:49.899926 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:49.900000 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":304,"threads_started":1}
I20260809 20:21:50.355682 24860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:50.354989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.3:38603)
0809 20:21:50.355401 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:50.355470 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":219,"threads_started":1}
I20260809 20:21:50.449832 24861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:50.449135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.1:38547)
0809 20:21:50.449550 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:50.449628 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":213,"threads_started":1}
I20260809 20:21:50.901749 24862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:50.901043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.2:51333)
0809 20:21:50.901466 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:50.901540 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":209,"threads_started":1}
I20260809 20:21:51.357393 24863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:51.356610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.3:45133)
0809 20:21:51.357097 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:51.357184 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":256,"threads_started":1}
I20260809 20:21:51.451730 24864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:51.450879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.1:53523)
0809 20:21:51.451428 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:51.451531 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":278,"threads_started":1}
I20260809 20:21:51.903533 24865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:51.902771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.2:52701)
0809 20:21:51.903284 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:51.903367 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":273,"threads_started":1}
I20260809 20:21:52.359119 24867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:52.358339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.3:41803)
0809 20:21:52.358873 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:52.358953 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":283,"threads_started":1}
I20260809 20:21:52.453226 24869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:52.452538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.1:36673)
0809 20:21:52.452976 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:52.453063 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":233,"threads_started":1}
I20260809 20:21:52.905063 24871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:52.904383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.2:56009)
0809 20:21:52.904819 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:52.904904 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":229,"threads_started":1}
I20260809 20:21:53.360781 24872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:53.360003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.3:39543)
0809 20:21:53.360478 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:53.360604 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":254,"threads_started":1}
I20260809 20:21:53.455118 24873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:53.454244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.1:47727)
0809 20:21:53.454815 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:53.454897 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":307,"threads_started":1}
I20260809 20:21:53.906788 24874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:53.905988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.2:40551)
0809 20:21:53.906481 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:53.906559 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":224,"threads_started":1}
I20260809 20:21:54.362563 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:54.361703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.3:44231)
0809 20:21:54.362259 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:54.362342 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":250,"threads_started":1}
I20260809 20:21:54.456768 24877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:54.456118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.1:48167)
0809 20:21:54.456532 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:54.456609 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":214,"threads_started":1}
I20260809 20:21:54.908384 24878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:54.907688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.2:49855)
0809 20:21:54.908131 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:54.908209 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":236,"threads_started":1}
I20260809 20:21:55.364415 24879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:55.363728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.3:55805)
0809 20:21:55.364165 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:55.364240 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":237,"threads_started":1}
I20260809 20:21:55.458475 24880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:55.457717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.1:36059)
0809 20:21:55.458193 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:55.458274 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":255,"threads_started":1}
I20260809 20:21:55.909916 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:55.909182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.2:46953)
0809 20:21:55.909653 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:55.909730 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":238,"threads_started":1}
I20260809 20:21:56.366065 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:56.365257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.3:45121)
0809 20:21:56.365767 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:56.365889 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":257,"threads_started":1}
I20260809 20:21:56.460115 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:56.459330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.1:43707)
0809 20:21:56.459850 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:56.459939 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":313,"threads_started":1}
I20260809 20:21:56.911813 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:56.911033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.2:59697)
0809 20:21:56.911517 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:56.911647 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":273,"threads_started":1}
I20260809 20:21:57.368065 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:57.367275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.3:48521)
0809 20:21:57.367828 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:57.367904 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":355,"threads_started":1}
I20260809 20:21:57.461820 24886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:57.460991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.1:43131)
0809 20:21:57.461542 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:57.461634 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":313,"threads_started":1}
I20260809 20:21:57.913304 24887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:57.912612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.2:41887)
0809 20:21:57.913040 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:57.913127 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":216,"threads_started":1}
I20260809 20:21:58.369846 24888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:58.368970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.3:42361)
0809 20:21:58.369502 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:58.369592 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":308,"threads_started":1}
I20260809 20:21:58.463438 24889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:58.462714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.1:43959)
0809 20:21:58.463180 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:58.463277 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":237,"threads_started":1}
I20260809 20:21:58.915016 24890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:58.914364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.2:50273)
0809 20:21:58.914776 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:58.914849 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":211,"threads_started":1}
I20260809 20:21:59.371776 24892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:59.371049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.3:44631)
0809 20:21:59.371471 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:59.371608 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":220,"threads_started":1}
I20260809 20:21:59.465040 24893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:59.464307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.1:34361)
0809 20:21:59.464769 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:59.464849 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":226,"threads_started":1}
I20260809 20:21:59.916756 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:21:59.916015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.2:35401)
0809 20:21:59.916463 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 20:21:59.916545 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":219,"threads_started":1}
I20260809 20:22:00.373413 24895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:00.372641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.3:35207)
0809 20:22:00.373115 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:00.373198 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":241,"threads_started":1}
I20260809 20:22:00.466573 24896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:00.465882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.1:41557)
0809 20:22:00.466315 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:00.466393 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":232,"threads_started":1}
I20260809 20:22:00.918452 24897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:00.917662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.2:56887)
0809 20:22:00.918077 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:00.918189 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":220,"threads_started":1}
I20260809 20:22:01.375041 24898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:01.374282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.3:48327)
0809 20:22:01.374770 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:01.374852 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":236,"threads_started":1}
I20260809 20:22:01.468359 24900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:01.467677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.1:57013)
0809 20:22:01.468098 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:01.468182 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":223,"threads_started":1}
I20260809 20:22:01.920274 24901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:01.919522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.2:42093)
0809 20:22:01.919985 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:01.920068 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":219,"threads_started":1}
I20260809 20:22:02.359500 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 24314
I20260809 20:22:02.377056 24903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:02.376203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:41859 (local address 127.5.61.3:41543)
0809 20:22:02.376734 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:02.376822 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.62:41859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":303,"threads_started":1}
I20260809 20:22:02.377606 24902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:02.360710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46050 (local address 127.5.61.1:44147)
0809 20:22:02.361291 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:02.361309 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:02.361357 (+ 48us) server_negotiation.cc:407] Connection header received
0809 20:22:02.363297 (+ 1940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:02.363307 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:02.363800 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:02.364090 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:02.365490 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:02.366735 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:02.368052 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:02.368724 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:02.368856 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:02.371403 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:02.371410 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:02.371425 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:02.371463 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:02.374291 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:02.374912 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:02.374919 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:02.374925 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:02.375078 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:02.375400 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:02.375403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:02.375405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:02.375961 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:02.376297 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:02.376759 (+ 462us) server_negotiation.cc:299] Negotiation successful
0809 20:22:02.377303 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":270,"threads_started":1}
I20260809 20:22:02.379097 24523 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:02.462532 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 24314
I20260809 20:22:02.482434 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 24532
I20260809 20:22:02.499446 24905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:02.483507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39242 (local address 127.5.61.2:37737)
0809 20:22:02.484173 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:02.484188 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:02.484225 (+ 37us) server_negotiation.cc:407] Connection header received
0809 20:22:02.485784 (+ 1559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:02.485794 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:02.486217 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:02.486552 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:02.487926 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:02.489131 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:02.490495 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:02.491149 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:02.491287 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:02.493932 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:02.493939 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:02.493953 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:02.493986 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:02.496672 (+ 2686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:02.497303 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:02.497309 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:02.497315 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:02.497480 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:02.497781 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:02.497784 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:02.497786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:02.498268 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:02.498527 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:02.498859 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 20:22:02.499169 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":343,"threads_started":1}
I20260809 20:22:02.500916 24682 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:02.586642 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 24532
I20260809 20:22:02.608567 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 24690
I20260809 20:22:02.625721 24907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:02.609475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41696 (local address 127.5.61.3:40389)
0809 20:22:02.610095 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:02.610109 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:02.610153 (+ 44us) server_negotiation.cc:407] Connection header received
0809 20:22:02.611866 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:02.611877 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:02.612458 (+ 581us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:02.612790 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:02.614016 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:02.615256 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:02.616605 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:02.617255 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:02.617391 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:02.619900 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:02.619908 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:02.619921 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:02.619970 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:02.622814 (+ 2844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:02.623488 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:02.623494 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:02.623500 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:02.623750 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:02.624077 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:02.624080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:02.624082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:02.624554 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:02.624770 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:02.625084 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 20:22:02.625416 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":304,"threads_started":1}
I20260809 20:22:02.627053 24842 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:02.711838 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 24690
2026-08-09T20:22:02Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64832 ms)
[----------] 30 tests from SecurityITest (299628 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:22:02 dist-test-slave-6k22 krb5kdc[24913](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:22:02 dist-test-slave-6k22 krb5kdc[24913](info): set up 2 sockets
Aug 09 20:22:02 dist-test-slave-6k22 krb5kdc[24913](info): commencing operation
krb5kdc: starting...
W20260809 20:22:03.910769 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:22:04 dist-test-slave-6k22 krb5kdc[24913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:22:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:22:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:22:04.195261 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:33235
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:46291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:33235
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:04.530239 24926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:04.530598 24926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:04.530752 24926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:04.537897 24926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:22:04.538034 24926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:04.538138 24926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:04.538233 24926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:22:04.538314 24926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:22:04.548077 24926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:46291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:33235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:33235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:04.549449 24926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:04.550810 24926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:04.562819 24932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:04.562932 24933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:04.563398 24936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:04.564512 24926 server_base.cc:1034] running on GCE node
I20260809 20:22:04.565369 24926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:04.566754 24926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:04.567956 24926 hybrid_clock.cc:648] HybridClock initialized: now 1786306924567908 us; error 44 us; skew 500 ppm
I20260809 20:22:04.568301 24926 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:04 dist-test-slave-6k22 krb5kdc[24913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306924, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:04.576201 24926 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:22:04.577764 24926 webserver.cc:460] Webserver started at http://127.5.61.62:40793/ using document root <none> and password file <none>
I20260809 20:22:04.578500 24926 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:04.578640 24926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:04.578946 24926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:04.581990 24926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "019a619017b44472a28688bb18705ccf"
format_stamp: "Formatted at 2026-08-09 20:22:04 on dist-test-slave-6k22"
server_key: "dff3c2af620a04555f0ceb34050dd972"
server_key_iv: "66e3174613190255f3486d29f5c9cb37"
server_key_version: "encryptionkey@0"
I20260809 20:22:04.582901 24926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "019a619017b44472a28688bb18705ccf"
format_stamp: "Formatted at 2026-08-09 20:22:04 on dist-test-slave-6k22"
server_key: "dff3c2af620a04555f0ceb34050dd972"
server_key_iv: "66e3174613190255f3486d29f5c9cb37"
server_key_version: "encryptionkey@0"
I20260809 20:22:04.588956 24926 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260809 20:22:04.592254 24945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:04.593549 24926 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:22:04.593742 24926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "019a619017b44472a28688bb18705ccf"
format_stamp: "Formatted at 2026-08-09 20:22:04 on dist-test-slave-6k22"
server_key: "dff3c2af620a04555f0ceb34050dd972"
server_key_iv: "66e3174613190255f3486d29f5c9cb37"
server_key_version: "encryptionkey@0"
I20260809 20:22:04.593932 24926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:04.608992 24926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:04.612644 24926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:04.612922 24926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:04.631270 24926 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:33235
I20260809 20:22:04.631317 25000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:33235 every 8 connection(s)
I20260809 20:22:04.632611 24926 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:22:04.636317 25001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:04.641917 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 24926
I20260809 20:22:04.642139 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:22:04.642814 5364 external_mini_cluster.cc:1428] Setting key f5d9e88548202e7f7526c11e2f27f358
I20260809 20:22:04.647518 25001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 019a619017b44472a28688bb18705ccf: Bootstrap starting.
I20260809 20:22:04.651294 25001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 019a619017b44472a28688bb18705ccf: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:04.652582 25001 log.cc:824] T 00000000000000000000000000000000 P 019a619017b44472a28688bb18705ccf: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:04.655733 25001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 019a619017b44472a28688bb18705ccf: No bootstrap required, opened a new log
Aug 09 20:22:04 dist-test-slave-6k22 krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:22:04.665097 25001 raft_consensus.cc:348] T 00000000000000000000000000000000 P 019a619017b44472a28688bb18705ccf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "019a619017b44472a28688bb18705ccf" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 33235 } }
I20260809 20:22:04.665552 25001 raft_consensus.cc:374] T 00000000000000000000000000000000 P 019a619017b44472a28688bb18705ccf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:04.665663 25001 raft_consensus.cc:729] T 00000000000000000000000000000000 P 019a619017b44472a28688bb18705ccf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019a619017b44472a28688bb18705ccf, State: Initialized, Role: FOLLOWER
I20260809 20:22:04.666350 25001 consensus_queue.cc:260] T 00000000000000000000000000000000 P 019a619017b44472a28688bb18705ccf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "019a619017b44472a28688bb18705ccf" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 33235 } }
I20260809 20:22:04.666671 25001 raft_consensus.cc:388] T 00000000000000000000000000000000 P 019a619017b44472a28688bb18705ccf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:22:04.666823 25001 raft_consensus.cc:482] T 00000000000000000000000000000000 P 019a619017b44472a28688bb18705ccf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:22:04.667053 25001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 019a619017b44472a28688bb18705ccf [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:04.668439 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:04.644754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36816 (local address 127.5.61.62:33235)
0809 20:22:04.645614 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:04.645629 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:04.645675 (+ 46us) server_negotiation.cc:407] Connection header received
0809 20:22:04.648129 (+ 2454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:04.648156 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:04.648614 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:04.649029 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:04.650473 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:04.652530 (+ 2057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:04.654399 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:04.655437 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:04.655686 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:04.659723 (+ 4037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:04.659733 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:04.659757 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:04.659805 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:04.663978 (+ 4173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:04.664659 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:04.664670 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:04.664681 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:04.664884 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:04.665242 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:04.665246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:04.665249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:04.665977 (+ 728us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:04.666262 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:04.666713 (+ 451us) server_negotiation.cc:299] Negotiation successful
0809 20:22:04.667172 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":366,"threads_started":1}
I20260809 20:22:04.670091 25001 raft_consensus.cc:504] T 00000000000000000000000000000000 P 019a619017b44472a28688bb18705ccf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "019a619017b44472a28688bb18705ccf" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 33235 } }
I20260809 20:22:04.670571 25001 leader_election.cc:302] T 00000000000000000000000000000000 P 019a619017b44472a28688bb18705ccf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 019a619017b44472a28688bb18705ccf; no voters:
I20260809 20:22:04.671211 25001 leader_election.cc:288] T 00000000000000000000000000000000 P 019a619017b44472a28688bb18705ccf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:22:04.671408 25006 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 019a619017b44472a28688bb18705ccf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:04.672258 25006 raft_consensus.cc:686] T 00000000000000000000000000000000 P 019a619017b44472a28688bb18705ccf [term 1 LEADER]: Becoming Leader. State: Replica: 019a619017b44472a28688bb18705ccf, State: Running, Role: LEADER
I20260809 20:22:04.672840 25006 consensus_queue.cc:237] T 00000000000000000000000000000000 P 019a619017b44472a28688bb18705ccf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "019a619017b44472a28688bb18705ccf" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 33235 } }
I20260809 20:22:04.673421 25001 sys_catalog.cc:564] T 00000000000000000000000000000000 P 019a619017b44472a28688bb18705ccf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:22:04.675781 25009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 019a619017b44472a28688bb18705ccf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 019a619017b44472a28688bb18705ccf. Latest consensus state: current_term: 1 leader_uuid: "019a619017b44472a28688bb18705ccf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "019a619017b44472a28688bb18705ccf" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 33235 } } }
I20260809 20:22:04.676244 25009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 019a619017b44472a28688bb18705ccf [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:04.675786 25007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 019a619017b44472a28688bb18705ccf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "019a619017b44472a28688bb18705ccf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "019a619017b44472a28688bb18705ccf" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 33235 } } }
I20260809 20:22:04.676467 25007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 019a619017b44472a28688bb18705ccf [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:04.684523 25013 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:22:04.689558 25013 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:22:04.699180 25013 catalog_manager.cc:1347] Generated new cluster ID: 0ce5e306d132475db73db8f028e18931
I20260809 20:22:04.699306 25013 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:22:04.709625 25013 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:22:04.710896 25013 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:22:04.720709 25013 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 019a619017b44472a28688bb18705ccf: Generated new TSK 0
I20260809 20:22:04.721467 25013 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.61.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
I20260809 20:22:04.803131 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:33235
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:46291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:05.155843 25031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:05.156150 25031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:05.156312 25031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:05.163416 25031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:05.163522 25031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:05.163750 25031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:22:05.173516 25031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:46291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:33235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:05.174897 25031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:05.176303 25031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:05.189317 25038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:05.189222 25039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:05.189222 25041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:05.189987 25031 server_base.cc:1034] running on GCE node
I20260809 20:22:05.190586 25031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:05.191468 25031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:05.192691 25031 hybrid_clock.cc:648] HybridClock initialized: now 1786306925192640 us; error 43 us; skew 500 ppm
I20260809 20:22:05.192993 25031 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:05 dist-test-slave-6k22 krb5kdc[24913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:05.196027 25031 init.cc:377] Logged in from keytab as kudu/127.5.61.1@KRBTEST.COM (short username kudu)
I20260809 20:22:05.197476 25031 webserver.cc:460] Webserver started at http://127.5.61.1:43687/ using document root <none> and password file <none>
I20260809 20:22:05.198122 25031 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:05.198256 25031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:05.198647 25031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:05.201505 25031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "1c95503922b14270b31235fd4dca66e3"
format_stamp: "Formatted at 2026-08-09 20:22:05 on dist-test-slave-6k22"
server_key: "7616e0cfa6d230ab91c1390d358a0af7"
server_key_iv: "eadb8caa71d55c8ca8f6f3c819647829"
server_key_version: "encryptionkey@0"
I20260809 20:22:05.202293 25031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "1c95503922b14270b31235fd4dca66e3"
format_stamp: "Formatted at 2026-08-09 20:22:05 on dist-test-slave-6k22"
server_key: "7616e0cfa6d230ab91c1390d358a0af7"
server_key_iv: "eadb8caa71d55c8ca8f6f3c819647829"
server_key_version: "encryptionkey@0"
I20260809 20:22:05.207243 25031 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 20:22:05.210561 25051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:05.211845 25031 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 20:22:05.212010 25031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "1c95503922b14270b31235fd4dca66e3"
format_stamp: "Formatted at 2026-08-09 20:22:05 on dist-test-slave-6k22"
server_key: "7616e0cfa6d230ab91c1390d358a0af7"
server_key_iv: "eadb8caa71d55c8ca8f6f3c819647829"
server_key_version: "encryptionkey@0"
I20260809 20:22:05.212200 25031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:05.227104 25031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:05.230633 25031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:05.230959 25031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:05.231887 25031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:22:05.233122 25031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:22:05.233222 25031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:05.233372 25031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:22:05.233445 25031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:05.265784 25031 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:43621
I20260809 20:22:05.265897 25179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:43621 every 8 connection(s)
I20260809 20:22:05.267086 25031 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:22:05.275234 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 25031
I20260809 20:22:05.275468 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:22:05.276204 5364 external_mini_cluster.cc:1428] Setting key 5c3ccae58cf81a81bbeb13271fa020dd
Aug 09 20:22:05 dist-test-slave-6k22 krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:22:05.287914 25183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:05.269543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:56587 (local address 127.5.61.62:33235)
0809 20:22:05.270016 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:05.270020 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:05.270778 (+ 758us) server_negotiation.cc:407] Connection header received
0809 20:22:05.272644 (+ 1866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:05.272650 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:05.272769 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:05.272886 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:05.274469 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:05.275625 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:05.277314 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:05.277971 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:05.278129 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:05.282131 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:05.282136 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:05.282139 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:05.282171 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:05.284787 (+ 2616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:05.285491 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:05.285498 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:05.285501 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:05.285624 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:05.286064 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:05.286070 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:05.286073 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:05.286325 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:05.286486 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:05.287198 (+ 712us) server_negotiation.cc:299] Negotiation successful
0809 20:22:05.287489 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":379,"thread_start_us":261,"threads_started":1}
I20260809 20:22:05.288879 25184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:05.269955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:33235 (local address 127.5.61.1:56587)
0809 20:22:05.270630 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:05.270666 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:05.272378 (+ 1712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:05.273120 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:05.273128 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:05.273713 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:22:05.274260 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:05.274277 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:05.275784 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:05.275789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:05.277152 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:05.277165 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:05.278203 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:05.278208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:05.278548 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:05.279252 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:22:05.279288 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:22:05.281915 (+ 2627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:22:05.284933 (+ 3018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:22:05.284942 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:22:05.284956 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:22:05.285357 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:22:05.285752 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:22:05.285757 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:22:05.285761 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:22:05.285954 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:22:05.286458 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:22:05.286465 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:22:05.286768 (+ 303us) client_negotiation.cc:769] Sending connection context
0809 20:22:05.287089 (+ 321us) client_negotiation.cc:240] Negotiation successful
0809 20:22:05.287613 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":282,"threads_started":1}
I20260809 20:22:05.290406 25180 heartbeater.cc:347] Connected to a master server at 127.5.61.62:33235
I20260809 20:22:05.290771 25180 heartbeater.cc:464] Registering TS with master...
I20260809 20:22:05.291505 25180 heartbeater.cc:511] Master 127.5.61.62:33235 requested a full tablet report, sending...
I20260809 20:22:05.293497 24963 ts_manager.cc:194] Registered new tserver with Master: 1c95503922b14270b31235fd4dca66e3 (127.5.61.1:43621)
I20260809 20:22:05.294878 24963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.1@KRBTEST.COM'} at 127.5.61.1:56587
WARNING: no policy specified for kudu/127.5.61.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
I20260809 20:22:05.342159 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:33235
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:46291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:05.681720 25188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:05.682008 25188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:05.682117 25188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:05.689065 25188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:05.689167 25188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:05.689280 25188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:22:05.698817 25188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:46291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:33235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:05.700069 25188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:05.701377 25188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:05.714164 25193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:05.714226 25195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:05.714262 25197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:05.715235 25188 server_base.cc:1034] running on GCE node
I20260809 20:22:05.715849 25188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:05.716781 25188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:05.717973 25188 hybrid_clock.cc:648] HybridClock initialized: now 1786306925717916 us; error 54 us; skew 500 ppm
I20260809 20:22:05.718268 25188 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:05 dist-test-slave-6k22 krb5kdc[24913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:05.721524 25188 init.cc:377] Logged in from keytab as kudu/127.5.61.2@KRBTEST.COM (short username kudu)
I20260809 20:22:05.722963 25188 webserver.cc:460] Webserver started at http://127.5.61.2:41523/ using document root <none> and password file <none>
I20260809 20:22:05.723688 25188 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:05.723829 25188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:05.724148 25188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:05.726951 25188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "969fec2e46e34dc3afeb70d1bbee26e4"
format_stamp: "Formatted at 2026-08-09 20:22:05 on dist-test-slave-6k22"
server_key: "53c6f00b17022de4f910b9b8f2ff7c07"
server_key_iv: "a737d9d06fa6d024831d1cd326c2b8ee"
server_key_version: "encryptionkey@0"
I20260809 20:22:05.727873 25188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "969fec2e46e34dc3afeb70d1bbee26e4"
format_stamp: "Formatted at 2026-08-09 20:22:05 on dist-test-slave-6k22"
server_key: "53c6f00b17022de4f910b9b8f2ff7c07"
server_key_iv: "a737d9d06fa6d024831d1cd326c2b8ee"
server_key_version: "encryptionkey@0"
I20260809 20:22:05.732925 25188 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 20:22:05.736331 25207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:05.737607 25188 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:22:05.737772 25188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "969fec2e46e34dc3afeb70d1bbee26e4"
format_stamp: "Formatted at 2026-08-09 20:22:05 on dist-test-slave-6k22"
server_key: "53c6f00b17022de4f910b9b8f2ff7c07"
server_key_iv: "a737d9d06fa6d024831d1cd326c2b8ee"
server_key_version: "encryptionkey@0"
I20260809 20:22:05.738008 25188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:05.755484 25188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:05.759780 25188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:05.760056 25188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:05.760972 25188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:22:05.762239 25188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:22:05.762311 25188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:05.762480 25188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:22:05.762552 25188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:05.795982 25188 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:32875
I20260809 20:22:05.796078 25343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:32875 every 8 connection(s)
I20260809 20:22:05.797333 25188 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:22:05.799118 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 25188
I20260809 20:22:05.799304 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:22:05.799901 5364 external_mini_cluster.cc:1428] Setting key 79ecda213d2807ced33a9392d8d5562d
Aug 09 20:22:05 dist-test-slave-6k22 krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306925, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:22:05.824412 25348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:05.801390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:49199 (local address 127.5.61.62:33235)
0809 20:22:05.807933 (+ 6543us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:05.807937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:05.807952 (+ 15us) server_negotiation.cc:407] Connection header received
0809 20:22:05.807997 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:05.808001 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:05.808055 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:05.808144 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:05.810300 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:05.811015 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:05.813490 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:05.813935 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:05.814042 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:05.818394 (+ 4352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:05.818399 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:05.818403 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:05.818434 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:05.821149 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:05.822082 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:05.822086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:05.822088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:05.822210 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:05.822679 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:05.822682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:05.822684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:05.822944 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:05.823051 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:05.823967 (+ 916us) server_negotiation.cc:299] Negotiation successful
0809 20:22:05.824130 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6454,"thread_start_us":6361,"threads_started":1}
I20260809 20:22:05.825992 25347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:05.800837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:33235 (local address 127.5.61.2:49199)
0809 20:22:05.801668 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:05.801721 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:05.804305 (+ 2584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:05.808446 (+ 4141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:05.808459 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:05.809252 (+ 793us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:22:05.810049 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:05.810076 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:05.811203 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:05.811211 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:05.813340 (+ 2129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:05.813369 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:05.814170 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:05.814179 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:05.814518 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:05.815289 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:22:05.815314 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:22:05.818162 (+ 2848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:22:05.821291 (+ 3129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:22:05.821303 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:22:05.821320 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:22:05.821965 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:22:05.822312 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:22:05.822319 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:22:05.822326 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:22:05.822572 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:22:05.823080 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:22:05.823090 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:22:05.823533 (+ 443us) client_negotiation.cc:769] Sending connection context
0809 20:22:05.823878 (+ 345us) client_negotiation.cc:240] Negotiation successful
0809 20:22:05.824466 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":317,"threads_started":1}
I20260809 20:22:05.827456 25344 heartbeater.cc:347] Connected to a master server at 127.5.61.62:33235
I20260809 20:22:05.827769 25344 heartbeater.cc:464] Registering TS with master...
I20260809 20:22:05.828496 25344 heartbeater.cc:511] Master 127.5.61.62:33235 requested a full tablet report, sending...
I20260809 20:22:05.830060 24963 ts_manager.cc:194] Registered new tserver with Master: 969fec2e46e34dc3afeb70d1bbee26e4 (127.5.61.2:32875)
I20260809 20:22:05.831406 24963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.2@KRBTEST.COM'} at 127.5.61.2:49199
WARNING: no policy specified for kudu/127.5.61.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
I20260809 20:22:05.868880 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:33235
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:46291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:06.213152 25352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:06.213452 25352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:06.213609 25352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:06.220870 25352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:06.221014 25352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:06.221179 25352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:22:06.230629 25352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:46291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:33235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:06.231885 25352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:06.233183 25352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:06.246170 25359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:06.246255 25361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:06.246277 25363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:06.247274 25352 server_base.cc:1034] running on GCE node
I20260809 20:22:06.247860 25352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:06.248721 25352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:06.249913 25352 hybrid_clock.cc:648] HybridClock initialized: now 1786306926249855 us; error 49 us; skew 500 ppm
I20260809 20:22:06.250244 25352 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:06 dist-test-slave-6k22 krb5kdc[24913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306926, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:06.253291 25352 init.cc:377] Logged in from keytab as kudu/127.5.61.3@KRBTEST.COM (short username kudu)
I20260809 20:22:06.254647 25352 webserver.cc:460] Webserver started at http://127.5.61.3:40093/ using document root <none> and password file <none>
I20260809 20:22:06.255301 25352 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:06.255436 25352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:06.255775 25352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:06.258648 25352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "804cb8f3765b463698e369c5b9237996"
format_stamp: "Formatted at 2026-08-09 20:22:06 on dist-test-slave-6k22"
server_key: "87c3ed903634025e7999e6dc24f11cc7"
server_key_iv: "e158f753e14e89ef63322e5c9b631de2"
server_key_version: "encryptionkey@0"
I20260809 20:22:06.259414 25352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "804cb8f3765b463698e369c5b9237996"
format_stamp: "Formatted at 2026-08-09 20:22:06 on dist-test-slave-6k22"
server_key: "87c3ed903634025e7999e6dc24f11cc7"
server_key_iv: "e158f753e14e89ef63322e5c9b631de2"
server_key_version: "encryptionkey@0"
I20260809 20:22:06.264318 25352 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 20:22:06.267913 25373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:06.269284 25352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 20:22:06.269457 25352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "804cb8f3765b463698e369c5b9237996"
format_stamp: "Formatted at 2026-08-09 20:22:06 on dist-test-slave-6k22"
server_key: "87c3ed903634025e7999e6dc24f11cc7"
server_key_iv: "e158f753e14e89ef63322e5c9b631de2"
server_key_version: "encryptionkey@0"
I20260809 20:22:06.269665 25352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:06.292514 25352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:06.296425 25352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:06.296715 25352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:06.297675 25180 heartbeater.cc:503] Master 127.5.61.62:33235 was elected leader, sending a full tablet report...
I20260809 20:22:06.297885 25352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:22:06.299753 25352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:22:06.299853 25352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:06.299980 25352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:22:06.300062 25352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:06.332933 25352 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:44169
I20260809 20:22:06.333029 25502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:44169 every 8 connection(s)
I20260809 20:22:06.334254 25352 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:22:06.337139 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 25352
I20260809 20:22:06.337334 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:22:06.337854 5364 external_mini_cluster.cc:1428] Setting key ade9c7ba1c1e287453b3ccf60edb36ed
Aug 09 20:22:06 dist-test-slave-6k22 krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306926, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:22:06.355002 25506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:06.336850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:39121 (local address 127.5.61.62:33235)
0809 20:22:06.337416 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:06.337421 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:06.338384 (+ 963us) server_negotiation.cc:407] Connection header received
0809 20:22:06.340856 (+ 2472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:06.340861 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:06.340962 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:06.341071 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:06.342726 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.343497 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.344994 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.345410 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.345514 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:06.348764 (+ 3250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:06.348768 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:06.348771 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:06.348802 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:06.351677 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:06.352558 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:06.352563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:06.352569 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:06.352717 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:06.353230 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:06.353233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:06.353235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:06.353567 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:06.353681 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:06.354574 (+ 893us) server_negotiation.cc:299] Negotiation successful
0809 20:22:06.354737 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":268,"threads_started":1}
I20260809 20:22:06.355808 25507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:06.337300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:33235 (local address 127.5.61.3:39121)
0809 20:22:06.338173 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:06.338226 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:06.340568 (+ 2342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:06.341306 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:06.341317 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:06.341974 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:22:06.342578 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:06.342596 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.343671 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.343675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:06.344869 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:06.344880 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.345591 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.345596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:06.345808 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:06.346375 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:22:06.346395 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:22:06.348579 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:22:06.351798 (+ 3219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:22:06.351804 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:22:06.351814 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:22:06.352413 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:22:06.352863 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:22:06.352867 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:22:06.352870 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:22:06.353109 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:22:06.353711 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:22:06.353717 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:22:06.353989 (+ 272us) client_negotiation.cc:769] Sending connection context
0809 20:22:06.354219 (+ 230us) client_negotiation.cc:240] Negotiation successful
0809 20:22:06.354667 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":317,"threads_started":1}
I20260809 20:22:06.357179 25503 heartbeater.cc:347] Connected to a master server at 127.5.61.62:33235
I20260809 20:22:06.357474 25503 heartbeater.cc:464] Registering TS with master...
I20260809 20:22:06.358148 25503 heartbeater.cc:511] Master 127.5.61.62:33235 requested a full tablet report, sending...
I20260809 20:22:06.359488 24963 ts_manager.cc:194] Registered new tserver with Master: 804cb8f3765b463698e369c5b9237996 (127.5.61.3:44169)
I20260809 20:22:06.360327 24963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.3@KRBTEST.COM'} at 127.5.61.3:39121
I20260809 20:22:06.365415 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 20:22:06.384606 25506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:06.370375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36832 (local address 127.5.61.62:33235)
0809 20:22:06.370569 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:06.370576 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:06.370947 (+ 371us) server_negotiation.cc:407] Connection header received
0809 20:22:06.371167 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:06.371174 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:06.371253 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:06.371361 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:06.373410 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.374372 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.375917 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.376610 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.376810 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:06.379244 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:06.379251 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:06.379255 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:06.379295 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:06.381892 (+ 2597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:06.382557 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:06.382563 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:06.382567 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:06.382694 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:06.383082 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:06.383087 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:06.383090 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:06.383411 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:06.383685 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:06.384013 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 20:22:06.384257 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 20:22:06.388484 24963 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:22:06.395443 24963 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:22:06.419075 25522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:06.407725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:44169 (local address 127.0.0.1:57432)
0809 20:22:06.409118 (+ 1393us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:06.409163 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:06.409327 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:06.411127 (+ 1800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:06.411133 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:06.411160 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:06.411653 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:06.411670 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.413669 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.413677 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:06.416027 (+ 2350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:06.416039 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.417933 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.417941 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:06.418153 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:06.418214 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 20:22:06.418339 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 20:22:06.418574 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1165,"mutex_wait_us":22,"thread_start_us":3474,"threads_started":1}
I20260809 20:22:06.419451 25521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:06.407783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:32875 (local address 127.0.0.1:46374)
0809 20:22:06.409916 (+ 2133us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:06.409936 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:06.410068 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:06.411853 (+ 1785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:06.411857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:06.411884 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:06.412350 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:06.412377 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.413717 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.413722 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:06.416351 (+ 2629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:06.416369 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.418485 (+ 2116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.418492 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:06.418684 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:06.418729 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 20:22:06.418843 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 20:22:06.418981 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1979,"mutex_wait_us":35,"thread_start_us":951,"threads_started":1}
I20260809 20:22:06.419642 25528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:06.410780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39656 (local address 127.5.61.1:43621)
0809 20:22:06.411387 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:06.411394 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:06.411422 (+ 28us) server_negotiation.cc:407] Connection header received
0809 20:22:06.411511 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:06.411519 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:06.411793 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:06.412621 (+ 828us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:06.412891 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.414472 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.416882 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.417867 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.418001 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:06.418239 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:06.419104 (+ 865us) server_negotiation.cc:299] Negotiation successful
0809 20:22:06.419286 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":559,"threads_started":1}
I20260809 20:22:06.419984 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:06.410173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:43621 (local address 127.0.0.1:39656)
0809 20:22:06.410652 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:06.410670 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:06.410794 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:06.412008 (+ 1214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:06.412012 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:06.412044 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:06.412427 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:06.412437 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.414640 (+ 2203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.414650 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:06.416708 (+ 2058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:06.416724 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.418396 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.418402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:06.418586 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:06.418615 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 20:22:06.418705 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 20:22:06.418833 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":25,"thread_start_us":839,"threads_started":1}
I20260809 20:22:06.420293 25525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:06.408389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57432 (local address 127.5.61.3:44169)
0809 20:22:06.410271 (+ 1882us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:06.410278 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:06.410304 (+ 26us) server_negotiation.cc:407] Connection header received
0809 20:22:06.410395 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:06.410400 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:06.410568 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:06.410766 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:06.411831 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.413527 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.416254 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.417721 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.417871 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:06.418114 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:06.419812 (+ 1698us) server_negotiation.cc:299] Negotiation successful
0809 20:22:06.419974 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1711,"thread_start_us":1532,"threads_started":1}
I20260809 20:22:06.420293 25523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:06.407826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46374 (local address 127.5.61.2:32875)
0809 20:22:06.409756 (+ 1930us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:06.409767 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:06.410199 (+ 432us) server_negotiation.cc:407] Connection header received
0809 20:22:06.410314 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:06.410323 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:06.411631 (+ 1308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:06.411831 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:06.412566 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.413582 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.416544 (+ 2962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.418016 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.418146 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:06.418280 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:06.419803 (+ 1523us) server_negotiation.cc:299] Negotiation successful
0809 20:22:06.419974 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1757,"thread_start_us":1599,"threads_started":1}
I20260809 20:22:06.423466 25264 tablet_service.cc:1467] Processing CreateTablet for tablet da9269864256406eb7f8efc343f1954a (DEFAULT_TABLE table=test-table [id=ce362bd0a4a947da890003eac54902da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:22:06.423844 25106 tablet_service.cc:1467] Processing CreateTablet for tablet da9269864256406eb7f8efc343f1954a (DEFAULT_TABLE table=test-table [id=ce362bd0a4a947da890003eac54902da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:22:06.424345 25431 tablet_service.cc:1467] Processing CreateTablet for tablet da9269864256406eb7f8efc343f1954a (DEFAULT_TABLE table=test-table [id=ce362bd0a4a947da890003eac54902da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:22:06.425067 25264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da9269864256406eb7f8efc343f1954a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:06.425293 25106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da9269864256406eb7f8efc343f1954a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:06.425755 25431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da9269864256406eb7f8efc343f1954a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:06.436053 25529 tablet_bootstrap.cc:492] T da9269864256406eb7f8efc343f1954a P 969fec2e46e34dc3afeb70d1bbee26e4: Bootstrap starting.
I20260809 20:22:06.439826 25530 tablet_bootstrap.cc:492] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3: Bootstrap starting.
I20260809 20:22:06.440400 25529 tablet_bootstrap.cc:654] T da9269864256406eb7f8efc343f1954a P 969fec2e46e34dc3afeb70d1bbee26e4: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:06.441351 25531 tablet_bootstrap.cc:492] T da9269864256406eb7f8efc343f1954a P 804cb8f3765b463698e369c5b9237996: Bootstrap starting.
I20260809 20:22:06.442368 25529 log.cc:824] T da9269864256406eb7f8efc343f1954a P 969fec2e46e34dc3afeb70d1bbee26e4: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:06.443362 25530 tablet_bootstrap.cc:654] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:06.444487 25531 tablet_bootstrap.cc:654] T da9269864256406eb7f8efc343f1954a P 804cb8f3765b463698e369c5b9237996: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:06.444562 25530 log.cc:824] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:06.445640 25531 log.cc:824] T da9269864256406eb7f8efc343f1954a P 804cb8f3765b463698e369c5b9237996: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:06.445988 25529 tablet_bootstrap.cc:492] T da9269864256406eb7f8efc343f1954a P 969fec2e46e34dc3afeb70d1bbee26e4: No bootstrap required, opened a new log
I20260809 20:22:06.446290 25529 ts_tablet_manager.cc:1397] T da9269864256406eb7f8efc343f1954a P 969fec2e46e34dc3afeb70d1bbee26e4: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260809 20:22:06.448189 25530 tablet_bootstrap.cc:492] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3: No bootstrap required, opened a new log
I20260809 20:22:06.448537 25530 ts_tablet_manager.cc:1397] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 20:22:06.453253 25529 raft_consensus.cc:348] T da9269864256406eb7f8efc343f1954a P 969fec2e46e34dc3afeb70d1bbee26e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "969fec2e46e34dc3afeb70d1bbee26e4" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 32875 } } peers { permanent_uuid: "804cb8f3765b463698e369c5b9237996" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 44169 } } peers { permanent_uuid: "1c95503922b14270b31235fd4dca66e3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43621 } }
I20260809 20:22:06.453608 25529 raft_consensus.cc:374] T da9269864256406eb7f8efc343f1954a P 969fec2e46e34dc3afeb70d1bbee26e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:06.453725 25529 raft_consensus.cc:729] T da9269864256406eb7f8efc343f1954a P 969fec2e46e34dc3afeb70d1bbee26e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 969fec2e46e34dc3afeb70d1bbee26e4, State: Initialized, Role: FOLLOWER
I20260809 20:22:06.454422 25529 consensus_queue.cc:260] T da9269864256406eb7f8efc343f1954a P 969fec2e46e34dc3afeb70d1bbee26e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "969fec2e46e34dc3afeb70d1bbee26e4" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 32875 } } peers { permanent_uuid: "804cb8f3765b463698e369c5b9237996" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 44169 } } peers { permanent_uuid: "1c95503922b14270b31235fd4dca66e3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43621 } }
I20260809 20:22:06.455592 25344 heartbeater.cc:503] Master 127.5.61.62:33235 was elected leader, sending a full tablet report...
I20260809 20:22:06.456207 25529 ts_tablet_manager.cc:1428] T da9269864256406eb7f8efc343f1954a P 969fec2e46e34dc3afeb70d1bbee26e4: Time spent starting tablet: real 0.010s user 0.005s sys 0.004s
I20260809 20:22:06.456377 25531 tablet_bootstrap.cc:492] T da9269864256406eb7f8efc343f1954a P 804cb8f3765b463698e369c5b9237996: No bootstrap required, opened a new log
I20260809 20:22:06.456616 25531 ts_tablet_manager.cc:1397] T da9269864256406eb7f8efc343f1954a P 804cb8f3765b463698e369c5b9237996: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.005s
I20260809 20:22:06.460291 25530 raft_consensus.cc:348] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "969fec2e46e34dc3afeb70d1bbee26e4" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 32875 } } peers { permanent_uuid: "804cb8f3765b463698e369c5b9237996" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 44169 } } peers { permanent_uuid: "1c95503922b14270b31235fd4dca66e3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43621 } }
I20260809 20:22:06.460728 25530 raft_consensus.cc:374] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:06.460831 25530 raft_consensus.cc:729] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c95503922b14270b31235fd4dca66e3, State: Initialized, Role: FOLLOWER
I20260809 20:22:06.461434 25530 consensus_queue.cc:260] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "969fec2e46e34dc3afeb70d1bbee26e4" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 32875 } } peers { permanent_uuid: "804cb8f3765b463698e369c5b9237996" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 44169 } } peers { permanent_uuid: "1c95503922b14270b31235fd4dca66e3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43621 } }
I20260809 20:22:06.463073 25530 ts_tablet_manager.cc:1428] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260809 20:22:06.465866 25531 raft_consensus.cc:348] T da9269864256406eb7f8efc343f1954a P 804cb8f3765b463698e369c5b9237996 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "969fec2e46e34dc3afeb70d1bbee26e4" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 32875 } } peers { permanent_uuid: "804cb8f3765b463698e369c5b9237996" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 44169 } } peers { permanent_uuid: "1c95503922b14270b31235fd4dca66e3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43621 } }
I20260809 20:22:06.466198 25531 raft_consensus.cc:374] T da9269864256406eb7f8efc343f1954a P 804cb8f3765b463698e369c5b9237996 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:06.466316 25531 raft_consensus.cc:729] T da9269864256406eb7f8efc343f1954a P 804cb8f3765b463698e369c5b9237996 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 804cb8f3765b463698e369c5b9237996, State: Initialized, Role: FOLLOWER
I20260809 20:22:06.466848 25531 consensus_queue.cc:260] T da9269864256406eb7f8efc343f1954a P 804cb8f3765b463698e369c5b9237996 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "969fec2e46e34dc3afeb70d1bbee26e4" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 32875 } } peers { permanent_uuid: "804cb8f3765b463698e369c5b9237996" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 44169 } } peers { permanent_uuid: "1c95503922b14270b31235fd4dca66e3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43621 } }
I20260809 20:22:06.467896 25503 heartbeater.cc:503] Master 127.5.61.62:33235 was elected leader, sending a full tablet report...
I20260809 20:22:06.468477 25531 ts_tablet_manager.cc:1428] T da9269864256406eb7f8efc343f1954a P 804cb8f3765b463698e369c5b9237996: Time spent starting tablet: real 0.012s user 0.004s sys 0.008s
W20260809 20:22:06.519372 25182 tablet.cc:2369] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:22:06.521915 25539 raft_consensus.cc:482] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:22:06.522200 25539 raft_consensus.cc:504] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "969fec2e46e34dc3afeb70d1bbee26e4" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 32875 } } peers { permanent_uuid: "804cb8f3765b463698e369c5b9237996" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 44169 } } peers { permanent_uuid: "1c95503922b14270b31235fd4dca66e3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43621 } }
I20260809 20:22:06.524009 25539 leader_election.cc:288] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 969fec2e46e34dc3afeb70d1bbee26e4 (127.5.61.2:32875), 804cb8f3765b463698e369c5b9237996 (127.5.61.3:44169)
I20260809 20:22:06.531610 25525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:06.524788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:47551 (local address 127.5.61.3:44169)
0809 20:22:06.524976 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:06.524982 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:06.525601 (+ 619us) server_negotiation.cc:407] Connection header received
0809 20:22:06.525761 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:06.525765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:06.525821 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:06.525916 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:06.526581 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.527312 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.529273 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.530435 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.530556 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:06.530654 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:06.531201 (+ 547us) server_negotiation.cc:299] Negotiation successful
0809 20:22:06.531330 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 20:22:06.531625 25523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:06.524661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:36145 (local address 127.5.61.2:32875)
0809 20:22:06.524905 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:06.524913 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:06.525494 (+ 581us) server_negotiation.cc:407] Connection header received
0809 20:22:06.525733 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:06.525739 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:06.525824 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:06.525918 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:06.526549 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.527236 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.529220 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.530415 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.530556 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:06.530649 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:06.531197 (+ 548us) server_negotiation.cc:299] Negotiation successful
0809 20:22:06.531336 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 20:22:06.531914 25542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:06.524721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:44169 (local address 127.5.61.1:47551)
0809 20:22:06.525510 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:06.525524 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:06.525649 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:06.525934 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:06.525938 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:06.525962 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:06.526402 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:06.526413 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.527452 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.527455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:06.529033 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:06.529065 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.530712 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.530717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:06.530983 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:06.531065 (+ 82us) client_negotiation.cc:769] Sending connection context
0809 20:22:06.531154 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 20:22:06.531354 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":180,"thread_start_us":491,"threads_started":1}
I20260809 20:22:06.531953 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:06.524212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:32875 (local address 127.5.61.1:36145)
0809 20:22:06.525371 (+ 1159us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:06.525393 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:06.525555 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:06.525935 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:06.525940 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:06.526028 (+ 88us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:06.526407 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:06.526416 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.527388 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.527395 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:06.529045 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:06.529061 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.530697 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.530704 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:06.530988 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:06.531062 (+ 74us) client_negotiation.cc:769] Sending connection context
0809 20:22:06.531156 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 20:22:06.531354 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1036,"mutex_wait_us":129,"spinlock_wait_cycles":20224,"thread_start_us":801,"threads_started":1}
I20260809 20:22:06.532943 25453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da9269864256406eb7f8efc343f1954a" candidate_uuid: "1c95503922b14270b31235fd4dca66e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "804cb8f3765b463698e369c5b9237996" is_pre_election: true
I20260809 20:22:06.533072 25292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da9269864256406eb7f8efc343f1954a" candidate_uuid: "1c95503922b14270b31235fd4dca66e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "969fec2e46e34dc3afeb70d1bbee26e4" is_pre_election: true
I20260809 20:22:06.533478 25453 raft_consensus.cc:2454] T da9269864256406eb7f8efc343f1954a P 804cb8f3765b463698e369c5b9237996 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c95503922b14270b31235fd4dca66e3 in term 0.
I20260809 20:22:06.533552 25292 raft_consensus.cc:2454] T da9269864256406eb7f8efc343f1954a P 969fec2e46e34dc3afeb70d1bbee26e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c95503922b14270b31235fd4dca66e3 in term 0.
I20260809 20:22:06.534176 25054 leader_election.cc:302] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1c95503922b14270b31235fd4dca66e3, 804cb8f3765b463698e369c5b9237996, 969fec2e46e34dc3afeb70d1bbee26e4; no voters:
I20260809 20:22:06.534612 25539 raft_consensus.cc:2781] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:22:06.534762 25539 raft_consensus.cc:482] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:22:06.534884 25539 raft_consensus.cc:3037] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:06.537120 25539 raft_consensus.cc:504] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "969fec2e46e34dc3afeb70d1bbee26e4" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 32875 } } peers { permanent_uuid: "804cb8f3765b463698e369c5b9237996" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 44169 } } peers { permanent_uuid: "1c95503922b14270b31235fd4dca66e3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43621 } }
I20260809 20:22:06.537763 25539 leader_election.cc:288] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3 [CANDIDATE]: Term 1 election: Requested vote from peers 969fec2e46e34dc3afeb70d1bbee26e4 (127.5.61.2:32875), 804cb8f3765b463698e369c5b9237996 (127.5.61.3:44169)
I20260809 20:22:06.538089 25292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da9269864256406eb7f8efc343f1954a" candidate_uuid: "1c95503922b14270b31235fd4dca66e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "969fec2e46e34dc3afeb70d1bbee26e4"
I20260809 20:22:06.538280 25292 raft_consensus.cc:3037] T da9269864256406eb7f8efc343f1954a P 969fec2e46e34dc3afeb70d1bbee26e4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:06.538224 25453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da9269864256406eb7f8efc343f1954a" candidate_uuid: "1c95503922b14270b31235fd4dca66e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "804cb8f3765b463698e369c5b9237996"
I20260809 20:22:06.538475 25453 raft_consensus.cc:3037] T da9269864256406eb7f8efc343f1954a P 804cb8f3765b463698e369c5b9237996 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:06.540637 25292 raft_consensus.cc:2454] T da9269864256406eb7f8efc343f1954a P 969fec2e46e34dc3afeb70d1bbee26e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c95503922b14270b31235fd4dca66e3 in term 1.
I20260809 20:22:06.540815 25453 raft_consensus.cc:2454] T da9269864256406eb7f8efc343f1954a P 804cb8f3765b463698e369c5b9237996 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c95503922b14270b31235fd4dca66e3 in term 1.
I20260809 20:22:06.541055 25056 leader_election.cc:302] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c95503922b14270b31235fd4dca66e3, 969fec2e46e34dc3afeb70d1bbee26e4; no voters:
I20260809 20:22:06.541359 25539 raft_consensus.cc:2781] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:06.541842 25539 raft_consensus.cc:686] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3 [term 1 LEADER]: Becoming Leader. State: Replica: 1c95503922b14270b31235fd4dca66e3, State: Running, Role: LEADER
I20260809 20:22:06.542294 25539 consensus_queue.cc:237] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "969fec2e46e34dc3afeb70d1bbee26e4" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 32875 } } peers { permanent_uuid: "804cb8f3765b463698e369c5b9237996" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 44169 } } peers { permanent_uuid: "1c95503922b14270b31235fd4dca66e3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43621 } }
I20260809 20:22:06.546608 24963 catalog_manager.cc:5495] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c95503922b14270b31235fd4dca66e3 (127.5.61.1). New cstate: current_term: 1 leader_uuid: "1c95503922b14270b31235fd4dca66e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "969fec2e46e34dc3afeb70d1bbee26e4" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 32875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "804cb8f3765b463698e369c5b9237996" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 44169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c95503922b14270b31235fd4dca66e3" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 43621 } health_report { overall_health: HEALTHY } } }
W20260809 20:22:06.549446 25346 tablet.cc:2369] T da9269864256406eb7f8efc343f1954a P 969fec2e46e34dc3afeb70d1bbee26e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:22:06.571095 25528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:06.565906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39666 (local address 127.5.61.1:43621)
0809 20:22:06.566114 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:06.566121 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:06.566140 (+ 19us) server_negotiation.cc:407] Connection header received
0809 20:22:06.566231 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:06.566235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:06.566306 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:06.566460 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:22:06.566845 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.567657 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.569073 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.569466 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.569593 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:06.570073 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:22:06.570541 (+ 468us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:22:06.570714 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:06.570765 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 20:22:06.570846 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 20:22:06.579180 25453 raft_consensus.cc:1261] T da9269864256406eb7f8efc343f1954a P 804cb8f3765b463698e369c5b9237996 [term 1 FOLLOWER]: Refusing update from remote peer 1c95503922b14270b31235fd4dca66e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:22:06.579181 25292 raft_consensus.cc:1261] T da9269864256406eb7f8efc343f1954a P 969fec2e46e34dc3afeb70d1bbee26e4 [term 1 FOLLOWER]: Refusing update from remote peer 1c95503922b14270b31235fd4dca66e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:22:06.580139 25543 consensus_queue.cc:1035] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "969fec2e46e34dc3afeb70d1bbee26e4" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 32875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:22:06.580464 25539 consensus_queue.cc:1035] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "804cb8f3765b463698e369c5b9237996" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 44169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 20:22:06.585916 25504 tablet.cc:2369] T da9269864256406eb7f8efc343f1954a P 804cb8f3765b463698e369c5b9237996: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:22:06.593521 25547 mvcc.cc:205] Tried to move back new op lower bound from 7316713171259346944 to 7316713171120807936. Current Snapshot: MvccSnapshot[applied={T|T < 7316713171259346944}]
I20260809 20:22:06.594453 25550 mvcc.cc:205] Tried to move back new op lower bound from 7316713171259346944 to 7316713171120807936. Current Snapshot: MvccSnapshot[applied={T|T < 7316713171259346944}]
I20260809 20:22:06.607163 24962 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:22:06.607494 24962 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:22:06.611097 24962 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 019a619017b44472a28688bb18705ccf: Sending DeleteTablet for 3 replicas of tablet da9269864256406eb7f8efc343f1954a
I20260809 20:22:06.612288 25431 tablet_service.cc:1514] Processing DeleteTablet for tablet da9269864256406eb7f8efc343f1954a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:22:06 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:57432
I20260809 20:22:06.612260 25264 tablet_service.cc:1514] Processing DeleteTablet for tablet da9269864256406eb7f8efc343f1954a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:22:06 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:46374
I20260809 20:22:06.612362 25106 tablet_service.cc:1514] Processing DeleteTablet for tablet da9269864256406eb7f8efc343f1954a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:22:06 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:39656
I20260809 20:22:06.614181 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 25031
I20260809 20:22:06.625594 25559 tablet_replica.cc:330] T da9269864256406eb7f8efc343f1954a P 804cb8f3765b463698e369c5b9237996: stopping tablet replica
I20260809 20:22:06.625921 25561 tablet_replica.cc:330] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3: stopping tablet replica
I20260809 20:22:06.626447 25561 raft_consensus.cc:2229] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:22:06.626948 25559 raft_consensus.cc:2229] T da9269864256406eb7f8efc343f1954a P 804cb8f3765b463698e369c5b9237996 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:22:06.627333 25561 raft_consensus.cc:2258] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:22:06.628235 25560 tablet_replica.cc:330] T da9269864256406eb7f8efc343f1954a P 969fec2e46e34dc3afeb70d1bbee26e4: stopping tablet replica
I20260809 20:22:06.628279 25559 raft_consensus.cc:2258] T da9269864256406eb7f8efc343f1954a P 804cb8f3765b463698e369c5b9237996 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:22:06.628713 25560 raft_consensus.cc:2229] T da9269864256406eb7f8efc343f1954a P 969fec2e46e34dc3afeb70d1bbee26e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:22:06.629256 25560 raft_consensus.cc:2258] T da9269864256406eb7f8efc343f1954a P 969fec2e46e34dc3afeb70d1bbee26e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:22:06.629724 25561 ts_tablet_manager.cc:1905] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:22:06.631484 25560 ts_tablet_manager.cc:1905] T da9269864256406eb7f8efc343f1954a P 969fec2e46e34dc3afeb70d1bbee26e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:22:06.636216 25559 ts_tablet_manager.cc:1905] T da9269864256406eb7f8efc343f1954a P 804cb8f3765b463698e369c5b9237996: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:22:06.637109 25561 ts_tablet_manager.cc:1918] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:22:06.637290 25561 log.cc:1196] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/da9269864256406eb7f8efc343f1954a
I20260809 20:22:06.637862 25561 ts_tablet_manager.cc:1939] T da9269864256406eb7f8efc343f1954a P 1c95503922b14270b31235fd4dca66e3: Deleting consensus metadata
I20260809 20:22:06.639698 24946 catalog_manager.cc:4841] TS 1c95503922b14270b31235fd4dca66e3 (127.5.61.1:43621): tablet da9269864256406eb7f8efc343f1954a (table test-table [id=ce362bd0a4a947da890003eac54902da]) successfully deleted
I20260809 20:22:06.643496 25560 ts_tablet_manager.cc:1918] T da9269864256406eb7f8efc343f1954a P 969fec2e46e34dc3afeb70d1bbee26e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:22:06.643603 25559 ts_tablet_manager.cc:1918] T da9269864256406eb7f8efc343f1954a P 804cb8f3765b463698e369c5b9237996: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:22:06.643725 25560 log.cc:1196] T da9269864256406eb7f8efc343f1954a P 969fec2e46e34dc3afeb70d1bbee26e4: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/da9269864256406eb7f8efc343f1954a
I20260809 20:22:06.643751 25559 log.cc:1196] T da9269864256406eb7f8efc343f1954a P 804cb8f3765b463698e369c5b9237996: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/da9269864256406eb7f8efc343f1954a
I20260809 20:22:06.644263 25559 ts_tablet_manager.cc:1939] T da9269864256406eb7f8efc343f1954a P 804cb8f3765b463698e369c5b9237996: Deleting consensus metadata
I20260809 20:22:06.644312 25560 ts_tablet_manager.cc:1939] T da9269864256406eb7f8efc343f1954a P 969fec2e46e34dc3afeb70d1bbee26e4: Deleting consensus metadata
Aug 09 20:22:06 dist-test-slave-6k22 krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.1@KRBTEST.COM
I20260809 20:22:06.645953 24947 catalog_manager.cc:4841] TS 804cb8f3765b463698e369c5b9237996 (127.5.61.3:44169): tablet da9269864256406eb7f8efc343f1954a (table test-table [id=ce362bd0a4a947da890003eac54902da]) successfully deleted
I20260809 20:22:06.646384 24949 catalog_manager.cc:4841] TS 969fec2e46e34dc3afeb70d1bbee26e4 (127.5.61.2:32875): tablet da9269864256406eb7f8efc343f1954a (table test-table [id=ce362bd0a4a947da890003eac54902da]) successfully deleted
I20260809 20:22:06.655145 25528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:06.615486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39670 (local address 127.5.61.1:43621)
0809 20:22:06.615679 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:06.615687 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:06.628046 (+ 12359us) server_negotiation.cc:407] Connection header received
0809 20:22:06.628338 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:06.628343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:06.628423 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:06.628515 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:06.630390 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.631506 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.633414 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.634102 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.634200 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:06.647324 (+ 13124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:06.647338 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:06.647366 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:06.647421 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:06.651648 (+ 4227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:06.652720 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:06.652735 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:06.652738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:06.652914 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:06.653346 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:06.653351 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:06.653355 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:06.653810 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:06.654099 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:06.654475 (+ 376us) server_negotiation.cc:299] Negotiation successful
0809 20:22:06.654726 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 20:22:06.655859 25175 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:06.751952 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 25031
I20260809 20:22:06.774683 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 25188
Aug 09 20:22:06 dist-test-slave-6k22 krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.2@KRBTEST.COM
I20260809 20:22:06.790213 25523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:06.775520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46376 (local address 127.5.61.2:32875)
0809 20:22:06.775744 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:06.775752 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:06.775773 (+ 21us) server_negotiation.cc:407] Connection header received
0809 20:22:06.776003 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:06.776009 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:06.776092 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:06.776198 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:06.777506 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.778209 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.779766 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.780246 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.780340 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:06.784707 (+ 4367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:06.784720 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:06.784746 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:06.784779 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:06.787346 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:06.788127 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:06.788133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:06.788136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:06.788268 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:06.788648 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:06.788652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:06.788654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:06.789050 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:06.789341 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:06.789668 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 20:22:06.789900 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 20:22:06.790868 25338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:06.878522 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 25188
I20260809 20:22:06.899708 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 25352
Aug 09 20:22:06 dist-test-slave-6k22 krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.3@KRBTEST.COM
I20260809 20:22:06.915221 25525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:06.900478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57446 (local address 127.5.61.3:44169)
0809 20:22:06.900689 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:06.900697 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:06.900717 (+ 20us) server_negotiation.cc:407] Connection header received
0809 20:22:06.900921 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:06.900927 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:06.901005 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:06.901120 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:06.902456 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.903222 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.904617 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:06.905018 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:06.905148 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:06.909774 (+ 4626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:06.909789 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:06.909805 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:06.909838 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:06.912444 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:06.913125 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:06.913133 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:06.913135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:06.913273 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:06.913641 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:06.913647 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:06.913650 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:06.914024 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:06.914315 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:06.914641 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 20:22:06.914878 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 20:22:06.915915 25497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:07.005808 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 25352
I20260809 20:22:07.025996 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 24926
I20260809 20:22:07.027137 24996 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:07.134647 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 24926
2026-08-09T20:22:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4424 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T20:22:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:22:07Z Disabled control of system clock
I20260809 20:22:07.220662 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:36293
--webserver_interface=127.5.61.62
--webserver_port=0
--builtin_ntp_servers=127.5.61.20:45301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:36293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 20:22:07.557080 25571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:07.557359 25571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:07.557439 25571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:07.564618 25571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:22:07.564721 25571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:07.564759 25571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:07.564813 25571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:22:07.564857 25571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:22:07.573834 25571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:45301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:36293
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:36293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:07.575084 25571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:07.576831 25571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:07.588721 25578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:07.588683 25579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:07.588977 25581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:07.589558 25571 server_base.cc:1034] running on GCE node
I20260809 20:22:07.590329 25571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:07.591693 25571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:07.592932 25571 hybrid_clock.cc:648] HybridClock initialized: now 1786306927592894 us; error 43 us; skew 500 ppm
I20260809 20:22:07.593415 25571 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:22:07.596828 25571 webserver.cc:460] Webserver started at http://127.5.61.62:37603/ using document root <none> and password file <none>
I20260809 20:22:07.597499 25571 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:07.597621 25571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:07.597930 25571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:07.600946 25571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "b94ae85dbec64d70ac73b36d25a2fded"
format_stamp: "Formatted at 2026-08-09 20:22:07 on dist-test-slave-6k22"
server_key: "74c3b15cf23f3f732d94397f9e5b9506"
server_key_iv: "90f8aaac4f62f4f100764bd1897ec816"
server_key_version: "encryptionkey@0"
I20260809 20:22:07.601850 25571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "b94ae85dbec64d70ac73b36d25a2fded"
format_stamp: "Formatted at 2026-08-09 20:22:07 on dist-test-slave-6k22"
server_key: "74c3b15cf23f3f732d94397f9e5b9506"
server_key_iv: "90f8aaac4f62f4f100764bd1897ec816"
server_key_version: "encryptionkey@0"
I20260809 20:22:07.606873 25571 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 20:22:07.610169 25587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:07.611418 25571 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 20:22:07.611645 25571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "b94ae85dbec64d70ac73b36d25a2fded"
format_stamp: "Formatted at 2026-08-09 20:22:07 on dist-test-slave-6k22"
server_key: "74c3b15cf23f3f732d94397f9e5b9506"
server_key_iv: "90f8aaac4f62f4f100764bd1897ec816"
server_key_version: "encryptionkey@0"
I20260809 20:22:07.611855 25571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:07.642601 25571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:07.643448 25571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:07.643761 25571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:07.662308 25571 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:36293
I20260809 20:22:07.662328 25645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:36293 every 8 connection(s)
I20260809 20:22:07.663686 25571 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:22:07.665203 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 25571
I20260809 20:22:07.665369 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:22:07.665881 5364 external_mini_cluster.cc:1428] Setting key 5ee99b76d815155907be1355b471bf2c
I20260809 20:22:07.667917 25646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:07.679824 25650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:07.667759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60636 (local address 127.5.61.62:36293)
0809 20:22:07.668813 (+ 1054us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:07.668824 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:07.668865 (+ 41us) server_negotiation.cc:407] Connection header received
0809 20:22:07.670391 (+ 1526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:07.670412 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:07.670852 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:07.671197 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:07.671880 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:07.673275 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:07.675405 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:07.676162 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:07.676290 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:07.677548 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:07.677555 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:07.677567 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:07.678130 (+ 563us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:07.678206 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:07.678232 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:07.678324 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:07.678569 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 20:22:07.678849 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":429,"threads_started":1}
I20260809 20:22:07.680915 25646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b94ae85dbec64d70ac73b36d25a2fded: Bootstrap starting.
I20260809 20:22:07.684036 25646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b94ae85dbec64d70ac73b36d25a2fded: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:07.685101 25646 log.cc:824] T 00000000000000000000000000000000 P b94ae85dbec64d70ac73b36d25a2fded: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:07.687853 25646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b94ae85dbec64d70ac73b36d25a2fded: No bootstrap required, opened a new log
I20260809 20:22:07.695052 25646 raft_consensus.cc:348] T 00000000000000000000000000000000 P b94ae85dbec64d70ac73b36d25a2fded [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b94ae85dbec64d70ac73b36d25a2fded" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 36293 } }
I20260809 20:22:07.695405 25646 raft_consensus.cc:374] T 00000000000000000000000000000000 P b94ae85dbec64d70ac73b36d25a2fded [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:07.695515 25646 raft_consensus.cc:729] T 00000000000000000000000000000000 P b94ae85dbec64d70ac73b36d25a2fded [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b94ae85dbec64d70ac73b36d25a2fded, State: Initialized, Role: FOLLOWER
I20260809 20:22:07.696082 25646 consensus_queue.cc:260] T 00000000000000000000000000000000 P b94ae85dbec64d70ac73b36d25a2fded [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b94ae85dbec64d70ac73b36d25a2fded" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 36293 } }
I20260809 20:22:07.696313 25646 raft_consensus.cc:388] T 00000000000000000000000000000000 P b94ae85dbec64d70ac73b36d25a2fded [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:22:07.696467 25646 raft_consensus.cc:482] T 00000000000000000000000000000000 P b94ae85dbec64d70ac73b36d25a2fded [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:22:07.696645 25646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b94ae85dbec64d70ac73b36d25a2fded [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:07.699121 25646 raft_consensus.cc:504] T 00000000000000000000000000000000 P b94ae85dbec64d70ac73b36d25a2fded [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b94ae85dbec64d70ac73b36d25a2fded" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 36293 } }
I20260809 20:22:07.699674 25646 leader_election.cc:302] T 00000000000000000000000000000000 P b94ae85dbec64d70ac73b36d25a2fded [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b94ae85dbec64d70ac73b36d25a2fded; no voters:
I20260809 20:22:07.700228 25646 leader_election.cc:288] T 00000000000000000000000000000000 P b94ae85dbec64d70ac73b36d25a2fded [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:22:07.700376 25652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b94ae85dbec64d70ac73b36d25a2fded [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:07.701081 25652 raft_consensus.cc:686] T 00000000000000000000000000000000 P b94ae85dbec64d70ac73b36d25a2fded [term 1 LEADER]: Becoming Leader. State: Replica: b94ae85dbec64d70ac73b36d25a2fded, State: Running, Role: LEADER
I20260809 20:22:07.701582 25652 consensus_queue.cc:237] T 00000000000000000000000000000000 P b94ae85dbec64d70ac73b36d25a2fded [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b94ae85dbec64d70ac73b36d25a2fded" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 36293 } }
I20260809 20:22:07.702306 25646 sys_catalog.cc:564] T 00000000000000000000000000000000 P b94ae85dbec64d70ac73b36d25a2fded [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:22:07.704121 25654 sys_catalog.cc:455] T 00000000000000000000000000000000 P b94ae85dbec64d70ac73b36d25a2fded [sys.catalog]: SysCatalogTable state changed. Reason: New leader b94ae85dbec64d70ac73b36d25a2fded. Latest consensus state: current_term: 1 leader_uuid: "b94ae85dbec64d70ac73b36d25a2fded" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b94ae85dbec64d70ac73b36d25a2fded" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 36293 } } }
I20260809 20:22:07.704541 25654 sys_catalog.cc:458] T 00000000000000000000000000000000 P b94ae85dbec64d70ac73b36d25a2fded [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:07.705925 25653 sys_catalog.cc:455] T 00000000000000000000000000000000 P b94ae85dbec64d70ac73b36d25a2fded [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b94ae85dbec64d70ac73b36d25a2fded" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b94ae85dbec64d70ac73b36d25a2fded" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 36293 } } }
I20260809 20:22:07.706283 25653 sys_catalog.cc:458] T 00000000000000000000000000000000 P b94ae85dbec64d70ac73b36d25a2fded [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:07.707630 25660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:22:07.713089 25660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:22:07.722460 25660 catalog_manager.cc:1347] Generated new cluster ID: a7b8a06962ab4b339f44f598d30c3ef7
I20260809 20:22:07.722577 25660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:22:07.754541 25660 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:22:07.755800 25660 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:22:07.769026 25660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b94ae85dbec64d70ac73b36d25a2fded: Generated new TSK 0
I20260809 20:22:07.769760 25660 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:22:07.782402 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:36293
--builtin_ntp_servers=127.5.61.20:45301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 20:22:08.127305 25677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:08.127682 25677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:08.127856 25677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:08.134863 25677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:08.135001 25677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:08.135188 25677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:22:08.144198 25677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:45301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:36293
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:08.145433 25677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:08.147217 25677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:08.160349 25682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:08.160343 25683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:08.160274 25686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:08.161505 25677 server_base.cc:1034] running on GCE node
I20260809 20:22:08.162132 25677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:08.163074 25677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:08.164296 25677 hybrid_clock.cc:648] HybridClock initialized: now 1786306928164261 us; error 57 us; skew 500 ppm
I20260809 20:22:08.164613 25677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:22:08.166761 25677 webserver.cc:460] Webserver started at http://127.5.61.1:39599/ using document root <none> and password file <none>
I20260809 20:22:08.167421 25677 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:08.167544 25677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:08.167894 25677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:08.170568 25677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "a0382dc3216e49caa8e396ab3536a147"
format_stamp: "Formatted at 2026-08-09 20:22:08 on dist-test-slave-6k22"
server_key: "936dca495d8055ff17e7cf3061d0bb12"
server_key_iv: "ab8e51be0cb4d32230f8ed1b4e495dbf"
server_key_version: "encryptionkey@0"
I20260809 20:22:08.171339 25677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "a0382dc3216e49caa8e396ab3536a147"
format_stamp: "Formatted at 2026-08-09 20:22:08 on dist-test-slave-6k22"
server_key: "936dca495d8055ff17e7cf3061d0bb12"
server_key_iv: "ab8e51be0cb4d32230f8ed1b4e495dbf"
server_key_version: "encryptionkey@0"
I20260809 20:22:08.176846 25677 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 20:22:08.181394 25695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:08.182690 25677 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 20:22:08.182857 25677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "a0382dc3216e49caa8e396ab3536a147"
format_stamp: "Formatted at 2026-08-09 20:22:08 on dist-test-slave-6k22"
server_key: "936dca495d8055ff17e7cf3061d0bb12"
server_key_iv: "ab8e51be0cb4d32230f8ed1b4e495dbf"
server_key_version: "encryptionkey@0"
I20260809 20:22:08.183069 25677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:08.206635 25677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:08.207393 25677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:08.207683 25677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:08.208667 25677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:22:08.210116 25677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:22:08.210225 25677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:08.210343 25677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:22:08.210446 25677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:08.242933 25677 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:46737
I20260809 20:22:08.243012 25827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:46737 every 8 connection(s)
I20260809 20:22:08.244275 25677 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:22:08.253424 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 25677
I20260809 20:22:08.253612 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:22:08.254055 5364 external_mini_cluster.cc:1428] Setting key b947e06377aa7fd53dcde51a4bfa9138
I20260809 20:22:08.256532 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:08.246742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:46759 (local address 127.5.61.62:36293)
0809 20:22:08.247248 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:08.247252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:08.248314 (+ 1062us) server_negotiation.cc:407] Connection header received
0809 20:22:08.250088 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:08.250094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:08.250178 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:08.250296 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:08.251335 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:08.252154 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:08.253714 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:08.254119 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:08.254253 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:08.255776 (+ 1523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:08.255781 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:08.255785 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:08.255802 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:08.255860 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:08.255876 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:08.255974 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:08.256205 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 20:22:08.256282 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":300,"threads_started":1}
I20260809 20:22:08.257258 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:36293
--builtin_ntp_servers=127.5.61.20:45301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 20:22:08.257907 25833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:08.247236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:36293 (local address 127.5.61.1:46759)
0809 20:22:08.248155 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:08.248190 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:08.249830 (+ 1640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:08.250528 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:08.250536 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:08.250624 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:22:08.251167 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:08.251182 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:08.252321 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:08.252325 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:08.253556 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:08.253574 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:08.254386 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:08.254395 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:08.254701 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:08.255425 (+ 724us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 20:22:08.255428 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:22:08.255471 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 20:22:08.255544 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:22:08.255980 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:22:08.255989 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:22:08.255995 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 20:22:08.256181 (+ 186us) client_negotiation.cc:240] Negotiation successful
0809 20:22:08.256559 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":319,"threads_started":1}
I20260809 20:22:08.259616 25829 heartbeater.cc:347] Connected to a master server at 127.5.61.62:36293
I20260809 20:22:08.259902 25829 heartbeater.cc:464] Registering TS with master...
I20260809 20:22:08.260592 25829 heartbeater.cc:511] Master 127.5.61.62:36293 requested a full tablet report, sending...
I20260809 20:22:08.262707 25606 ts_manager.cc:194] Registered new tserver with Master: a0382dc3216e49caa8e396ab3536a147 (127.5.61.1:46737)
I20260809 20:22:08.264822 25606 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.61.1:46759
W20260809 20:22:08.603765 25834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:08.604035 25834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:08.604212 25834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:08.611377 25834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:08.611527 25834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:08.611757 25834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:22:08.621043 25834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:45301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:36293
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:08.622385 25834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:08.624189 25834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:08.637625 25843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:08.637603 25841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:08.637807 25840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:08.638196 25834 server_base.cc:1034] running on GCE node
I20260809 20:22:08.639047 25834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:08.640010 25834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:08.641238 25834 hybrid_clock.cc:648] HybridClock initialized: now 1786306928641177 us; error 55 us; skew 500 ppm
I20260809 20:22:08.641547 25834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:22:08.643847 25834 webserver.cc:460] Webserver started at http://127.5.61.2:33111/ using document root <none> and password file <none>
I20260809 20:22:08.644539 25834 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:08.644667 25834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:08.644989 25834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:08.647987 25834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "28fb493e03bf4b0ebb31fab310774288"
format_stamp: "Formatted at 2026-08-09 20:22:08 on dist-test-slave-6k22"
server_key: "e7c767e3f7526c674b92fd7b1e1fc28a"
server_key_iv: "5155bbb14230512113510c9726dc9c70"
server_key_version: "encryptionkey@0"
I20260809 20:22:08.648811 25834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "28fb493e03bf4b0ebb31fab310774288"
format_stamp: "Formatted at 2026-08-09 20:22:08 on dist-test-slave-6k22"
server_key: "e7c767e3f7526c674b92fd7b1e1fc28a"
server_key_iv: "5155bbb14230512113510c9726dc9c70"
server_key_version: "encryptionkey@0"
I20260809 20:22:08.653930 25834 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 20:22:08.657472 25850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:08.658797 25834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 20:22:08.658982 25834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "28fb493e03bf4b0ebb31fab310774288"
format_stamp: "Formatted at 2026-08-09 20:22:08 on dist-test-slave-6k22"
server_key: "e7c767e3f7526c674b92fd7b1e1fc28a"
server_key_iv: "5155bbb14230512113510c9726dc9c70"
server_key_version: "encryptionkey@0"
I20260809 20:22:08.659251 25834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:08.681016 25834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:08.681967 25834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:08.682243 25834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:08.683349 25834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:22:08.685066 25834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:22:08.685160 25834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:08.685240 25834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:22:08.685297 25834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:08.721709 25834 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:40727
I20260809 20:22:08.721701 25981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:40727 every 8 connection(s)
I20260809 20:22:08.723166 25834 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:22:08.728732 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 25834
I20260809 20:22:08.733492 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:22:08.734123 5364 external_mini_cluster.cc:1428] Setting key cded4dc9dd78464d61b8d7513435e8a0
I20260809 20:22:08.737043 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:36293
--builtin_ntp_servers=127.5.61.20:45301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 20:22:08.737988 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:08.726102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:55079 (local address 127.5.61.62:36293)
0809 20:22:08.726315 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:08.726321 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:08.727752 (+ 1431us) server_negotiation.cc:407] Connection header received
0809 20:22:08.730398 (+ 2646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:08.730402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:08.730469 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:08.730558 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:08.732123 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:08.732836 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:08.735047 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:08.735490 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:08.735681 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:08.737124 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:08.737129 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:08.737132 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:08.737149 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:08.737199 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:08.737214 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:08.737306 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:08.737538 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 20:22:08.737637 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":70}
I20260809 20:22:08.739002 25984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:08.726437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:36293 (local address 127.5.61.2:55079)
0809 20:22:08.727552 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:08.727641 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:08.730089 (+ 2448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:08.730856 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:08.730878 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:08.731013 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:22:08.731943 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:08.731973 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:08.733008 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:08.733014 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:08.734914 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:08.734927 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:08.735744 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:08.735753 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:08.736045 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:08.736806 (+ 761us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 20:22:08.736809 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:22:08.736848 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 20:22:08.736925 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:22:08.737326 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:22:08.737338 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:22:08.737346 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 20:22:08.737527 (+ 181us) client_negotiation.cc:240] Negotiation successful
0809 20:22:08.737934 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":395,"threads_started":1}
I20260809 20:22:08.740456 25982 heartbeater.cc:347] Connected to a master server at 127.5.61.62:36293
I20260809 20:22:08.740763 25982 heartbeater.cc:464] Registering TS with master...
I20260809 20:22:08.741465 25982 heartbeater.cc:511] Master 127.5.61.62:36293 requested a full tablet report, sending...
I20260809 20:22:08.742904 25606 ts_manager.cc:194] Registered new tserver with Master: 28fb493e03bf4b0ebb31fab310774288 (127.5.61.2:40727)
I20260809 20:22:08.744130 25606 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.61.2:55079
W20260809 20:22:09.075579 25986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:09.075845 25986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:09.075932 25986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:09.083003 25986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:09.083101 25986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:09.083218 25986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:22:09.092156 25986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:45301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:36293
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:09.093381 25986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:09.095047 25986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:09.108137 25992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:09.108166 25991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:09.108083 25995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:09.108671 25986 server_base.cc:1034] running on GCE node
I20260809 20:22:09.109339 25986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:09.110249 25986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:09.111479 25986 hybrid_clock.cc:648] HybridClock initialized: now 1786306929111436 us; error 51 us; skew 500 ppm
I20260809 20:22:09.111810 25986 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:22:09.113968 25986 webserver.cc:460] Webserver started at http://127.5.61.3:40367/ using document root <none> and password file <none>
I20260809 20:22:09.114646 25986 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:09.114771 25986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:09.115116 25986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:09.117969 25986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "d180be1f19c5447c8571bab587f3b515"
format_stamp: "Formatted at 2026-08-09 20:22:09 on dist-test-slave-6k22"
server_key: "2e6b786df004cc3ce65688d8f724bdbe"
server_key_iv: "497405efc961fd1d26832743e2a4ed80"
server_key_version: "encryptionkey@0"
I20260809 20:22:09.118772 25986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "d180be1f19c5447c8571bab587f3b515"
format_stamp: "Formatted at 2026-08-09 20:22:09 on dist-test-slave-6k22"
server_key: "2e6b786df004cc3ce65688d8f724bdbe"
server_key_iv: "497405efc961fd1d26832743e2a4ed80"
server_key_version: "encryptionkey@0"
I20260809 20:22:09.123824 25986 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 20:22:09.127240 26004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:09.128608 25986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:22:09.128738 25986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "d180be1f19c5447c8571bab587f3b515"
format_stamp: "Formatted at 2026-08-09 20:22:09 on dist-test-slave-6k22"
server_key: "2e6b786df004cc3ce65688d8f724bdbe"
server_key_iv: "497405efc961fd1d26832743e2a4ed80"
server_key_version: "encryptionkey@0"
I20260809 20:22:09.128909 25986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:09.146070 25986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:09.146773 25986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:09.147028 25986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:09.147930 25986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:22:09.149247 25986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:22:09.149355 25986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:09.149502 25986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:22:09.149559 25986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:09.182811 25986 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:42109
I20260809 20:22:09.182863 26139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:42109 every 8 connection(s)
I20260809 20:22:09.184201 25986 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:22:09.187283 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 25986
I20260809 20:22:09.187456 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:22:09.188108 5364 external_mini_cluster.cc:1428] Setting key 04415247da2ee616cc7ca2f2dd0e9794
I20260809 20:22:09.197098 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:09.186934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:40735 (local address 127.5.61.62:36293)
0809 20:22:09.187151 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:09.187158 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:09.188809 (+ 1651us) server_negotiation.cc:407] Connection header received
0809 20:22:09.190670 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:09.190676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:09.190761 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:09.190875 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:09.192001 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.192754 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.194288 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.194708 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.194838 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:09.196240 (+ 1402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:09.196246 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:09.196251 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:09.196274 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:09.196307 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:09.196319 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:09.196418 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:09.196631 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 20:22:09.196725 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 20:22:09.198043 26142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:09.187376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:36293 (local address 127.5.61.3:40735)
0809 20:22:09.188645 (+ 1269us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:09.188699 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:09.190414 (+ 1715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:09.191104 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:09.191112 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:09.191194 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:22:09.191841 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:09.191866 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.192910 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.192914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:09.194134 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:09.194146 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.194933 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.194940 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:09.195151 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:09.195922 (+ 771us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 20:22:09.195925 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:22:09.195960 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 20:22:09.196060 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:22:09.196417 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:22:09.196432 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:22:09.196442 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 20:22:09.196603 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 20:22:09.196966 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":531,"threads_started":1}
I20260809 20:22:09.199277 26140 heartbeater.cc:347] Connected to a master server at 127.5.61.62:36293
I20260809 20:22:09.199604 26140 heartbeater.cc:464] Registering TS with master...
I20260809 20:22:09.200299 26140 heartbeater.cc:511] Master 127.5.61.62:36293 requested a full tablet report, sending...
I20260809 20:22:09.201697 25606 ts_manager.cc:194] Registered new tserver with Master: d180be1f19c5447c8571bab587f3b515 (127.5.61.3:42109)
I20260809 20:22:09.202751 25606 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.61.3:40735
I20260809 20:22:09.202828 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 20:22:09.212837 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:09.207399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34878 (local address 127.5.61.62:36293)
0809 20:22:09.207630 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:09.207639 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:09.208018 (+ 379us) server_negotiation.cc:407] Connection header received
0809 20:22:09.208133 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:09.208136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:09.208202 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:09.208276 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:09.208758 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.209426 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.210721 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.211101 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.211187 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:09.212215 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:09.212219 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:09.212222 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:09.212239 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:09.212268 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:09.212279 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:09.212378 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:09.212475 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 20:22:09.212539 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 20:22:09.216192 25606 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34878:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:22:09.222782 25606 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:22:09.257700 26155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:09.234386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:40727 (local address 127.0.0.1:40870)
0809 20:22:09.247942 (+ 13556us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:09.247980 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:09.248235 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:09.249215 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:09.249234 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:09.249262 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:09.249549 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:09.249560 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.251551 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.251579 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:09.254619 (+ 3040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:09.254631 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.256689 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.256695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:09.256895 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:09.256957 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 20:22:09.257096 (+ 139us) client_negotiation.cc:240] Negotiation successful
0809 20:22:09.257337 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13291,"thread_start_us":13125,"threads_started":1}
I20260809 20:22:09.257717 26156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:09.234649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40870 (local address 127.5.61.2:40727)
0809 20:22:09.247174 (+ 12525us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:09.247183 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:09.248092 (+ 909us) server_negotiation.cc:407] Connection header received
0809 20:22:09.248493 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:09.248501 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:09.248725 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:09.248929 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:09.249750 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.251302 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.254842 (+ 3540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.256505 (+ 1663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.256636 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:09.256897 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:09.257201 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 20:22:09.257380 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12352,"thread_start_us":12530,"threads_started":1}
I20260809 20:22:09.262400 26157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:09.234749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:46737 (local address 127.0.0.1:51064)
0809 20:22:09.254263 (+ 19514us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:09.254281 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:09.254389 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:09.255046 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:09.255051 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:09.255080 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:09.255656 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:09.255672 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.257665 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.257672 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:09.259469 (+ 1797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:09.259481 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.261692 (+ 2211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.261698 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:09.261855 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:09.261898 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 20:22:09.261985 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 20:22:09.262104 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19359,"thread_start_us":20487,"threads_started":1}
I20260809 20:22:09.262629 26158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:09.235411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51064 (local address 127.5.61.1:46737)
0809 20:22:09.250696 (+ 15285us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:09.250708 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:09.254510 (+ 3802us) server_negotiation.cc:407] Connection header received
0809 20:22:09.254605 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:09.254612 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:09.254849 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:09.255036 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:09.255931 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.257500 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.259733 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.261149 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.261256 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:09.261514 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:09.262075 (+ 561us) server_negotiation.cc:299] Negotiation successful
0809 20:22:09.262260 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15069,"thread_start_us":15283,"threads_started":1}
I20260809 20:22:09.262661 25905 tablet_service.cc:1467] Processing CreateTablet for tablet 99d31f9201d245fcb30496eea4d20307 (DEFAULT_TABLE table=test-table [id=8836e98631214f64a6101b732f5feafc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:22:09.263918 26160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:09.251766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:42109 (local address 127.0.0.1:51030)
0809 20:22:09.256120 (+ 4354us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:09.256137 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:09.256268 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:09.256983 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:09.256988 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:09.257023 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:09.257413 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:09.257424 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.259909 (+ 2485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.259917 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:09.261159 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:09.261169 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.263151 (+ 1982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.263155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:09.263291 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:09.263334 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 20:22:09.263422 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 20:22:09.263539 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4238,"thread_start_us":4090,"threads_started":1}
I20260809 20:22:09.264056 26159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:09.248827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51030 (local address 127.5.61.3:42109)
0809 20:22:09.255085 (+ 6258us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:09.255093 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:09.256245 (+ 1152us) server_negotiation.cc:407] Connection header received
0809 20:22:09.256435 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:09.256460 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:09.256660 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:09.256829 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:09.257633 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.259773 (+ 2140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.261339 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.262606 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.262755 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:09.263049 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:09.263465 (+ 416us) server_negotiation.cc:299] Negotiation successful
0809 20:22:09.263702 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6079,"thread_start_us":5910,"threads_started":1}
I20260809 20:22:09.264197 25905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99d31f9201d245fcb30496eea4d20307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:09.267518 25829 heartbeater.cc:503] Master 127.5.61.62:36293 was elected leader, sending a full tablet report...
I20260809 20:22:09.267148 25754 tablet_service.cc:1467] Processing CreateTablet for tablet 99d31f9201d245fcb30496eea4d20307 (DEFAULT_TABLE table=test-table [id=8836e98631214f64a6101b732f5feafc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:22:09.267670 26058 tablet_service.cc:1467] Processing CreateTablet for tablet 99d31f9201d245fcb30496eea4d20307 (DEFAULT_TABLE table=test-table [id=8836e98631214f64a6101b732f5feafc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:22:09.268594 25754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99d31f9201d245fcb30496eea4d20307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:09.269021 26058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99d31f9201d245fcb30496eea4d20307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:09.279412 26161 tablet_bootstrap.cc:492] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147: Bootstrap starting.
I20260809 20:22:09.283306 26161 tablet_bootstrap.cc:654] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:09.284614 26161 log.cc:824] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:09.284650 26163 tablet_bootstrap.cc:492] T 99d31f9201d245fcb30496eea4d20307 P d180be1f19c5447c8571bab587f3b515: Bootstrap starting.
I20260809 20:22:09.285465 26162 tablet_bootstrap.cc:492] T 99d31f9201d245fcb30496eea4d20307 P 28fb493e03bf4b0ebb31fab310774288: Bootstrap starting.
I20260809 20:22:09.288609 26162 tablet_bootstrap.cc:654] T 99d31f9201d245fcb30496eea4d20307 P 28fb493e03bf4b0ebb31fab310774288: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:09.288918 26163 tablet_bootstrap.cc:654] T 99d31f9201d245fcb30496eea4d20307 P d180be1f19c5447c8571bab587f3b515: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:09.290366 26162 log.cc:824] T 99d31f9201d245fcb30496eea4d20307 P 28fb493e03bf4b0ebb31fab310774288: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:09.290419 26163 log.cc:824] T 99d31f9201d245fcb30496eea4d20307 P d180be1f19c5447c8571bab587f3b515: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:09.293286 26162 tablet_bootstrap.cc:492] T 99d31f9201d245fcb30496eea4d20307 P 28fb493e03bf4b0ebb31fab310774288: No bootstrap required, opened a new log
I20260809 20:22:09.293560 26162 ts_tablet_manager.cc:1397] T 99d31f9201d245fcb30496eea4d20307 P 28fb493e03bf4b0ebb31fab310774288: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 20:22:09.293645 26161 tablet_bootstrap.cc:492] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147: No bootstrap required, opened a new log
I20260809 20:22:09.293856 26163 tablet_bootstrap.cc:492] T 99d31f9201d245fcb30496eea4d20307 P d180be1f19c5447c8571bab587f3b515: No bootstrap required, opened a new log
I20260809 20:22:09.293941 26161 ts_tablet_manager.cc:1397] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.006s
I20260809 20:22:09.294112 26163 ts_tablet_manager.cc:1397] T 99d31f9201d245fcb30496eea4d20307 P d180be1f19c5447c8571bab587f3b515: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 20:22:09.301045 26162 raft_consensus.cc:348] T 99d31f9201d245fcb30496eea4d20307 P 28fb493e03bf4b0ebb31fab310774288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28fb493e03bf4b0ebb31fab310774288" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 40727 } } peers { permanent_uuid: "a0382dc3216e49caa8e396ab3536a147" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 46737 } } peers { permanent_uuid: "d180be1f19c5447c8571bab587f3b515" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 42109 } }
I20260809 20:22:09.302134 26162 raft_consensus.cc:374] T 99d31f9201d245fcb30496eea4d20307 P 28fb493e03bf4b0ebb31fab310774288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:09.302294 26162 raft_consensus.cc:729] T 99d31f9201d245fcb30496eea4d20307 P 28fb493e03bf4b0ebb31fab310774288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28fb493e03bf4b0ebb31fab310774288, State: Initialized, Role: FOLLOWER
I20260809 20:22:09.302917 26162 consensus_queue.cc:260] T 99d31f9201d245fcb30496eea4d20307 P 28fb493e03bf4b0ebb31fab310774288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28fb493e03bf4b0ebb31fab310774288" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 40727 } } peers { permanent_uuid: "a0382dc3216e49caa8e396ab3536a147" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 46737 } } peers { permanent_uuid: "d180be1f19c5447c8571bab587f3b515" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 42109 } }
I20260809 20:22:09.304215 25982 heartbeater.cc:503] Master 127.5.61.62:36293 was elected leader, sending a full tablet report...
I20260809 20:22:09.304979 26162 ts_tablet_manager.cc:1428] T 99d31f9201d245fcb30496eea4d20307 P 28fb493e03bf4b0ebb31fab310774288: Time spent starting tablet: real 0.011s user 0.011s sys 0.001s
I20260809 20:22:09.304771 26161 raft_consensus.cc:348] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28fb493e03bf4b0ebb31fab310774288" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 40727 } } peers { permanent_uuid: "a0382dc3216e49caa8e396ab3536a147" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 46737 } } peers { permanent_uuid: "d180be1f19c5447c8571bab587f3b515" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 42109 } }
I20260809 20:22:09.304814 26163 raft_consensus.cc:348] T 99d31f9201d245fcb30496eea4d20307 P d180be1f19c5447c8571bab587f3b515 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28fb493e03bf4b0ebb31fab310774288" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 40727 } } peers { permanent_uuid: "a0382dc3216e49caa8e396ab3536a147" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 46737 } } peers { permanent_uuid: "d180be1f19c5447c8571bab587f3b515" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 42109 } }
I20260809 20:22:09.305210 26161 raft_consensus.cc:374] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:09.305210 26163 raft_consensus.cc:374] T 99d31f9201d245fcb30496eea4d20307 P d180be1f19c5447c8571bab587f3b515 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:09.305318 26163 raft_consensus.cc:729] T 99d31f9201d245fcb30496eea4d20307 P d180be1f19c5447c8571bab587f3b515 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d180be1f19c5447c8571bab587f3b515, State: Initialized, Role: FOLLOWER
I20260809 20:22:09.305351 26161 raft_consensus.cc:729] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0382dc3216e49caa8e396ab3536a147, State: Initialized, Role: FOLLOWER
I20260809 20:22:09.305954 26161 consensus_queue.cc:260] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28fb493e03bf4b0ebb31fab310774288" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 40727 } } peers { permanent_uuid: "a0382dc3216e49caa8e396ab3536a147" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 46737 } } peers { permanent_uuid: "d180be1f19c5447c8571bab587f3b515" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 42109 } }
I20260809 20:22:09.305985 26163 consensus_queue.cc:260] T 99d31f9201d245fcb30496eea4d20307 P d180be1f19c5447c8571bab587f3b515 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28fb493e03bf4b0ebb31fab310774288" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 40727 } } peers { permanent_uuid: "a0382dc3216e49caa8e396ab3536a147" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 46737 } } peers { permanent_uuid: "d180be1f19c5447c8571bab587f3b515" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 42109 } }
I20260809 20:22:09.307056 26140 heartbeater.cc:503] Master 127.5.61.62:36293 was elected leader, sending a full tablet report...
I20260809 20:22:09.308139 26163 ts_tablet_manager.cc:1428] T 99d31f9201d245fcb30496eea4d20307 P d180be1f19c5447c8571bab587f3b515: Time spent starting tablet: real 0.014s user 0.006s sys 0.009s
I20260809 20:22:09.309747 26161 ts_tablet_manager.cc:1428] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147: Time spent starting tablet: real 0.016s user 0.014s sys 0.002s
I20260809 20:22:09.405390 26169 raft_consensus.cc:482] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:22:09.405726 26169 raft_consensus.cc:504] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28fb493e03bf4b0ebb31fab310774288" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 40727 } } peers { permanent_uuid: "a0382dc3216e49caa8e396ab3536a147" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 46737 } } peers { permanent_uuid: "d180be1f19c5447c8571bab587f3b515" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 42109 } }
I20260809 20:22:09.407662 26169 leader_election.cc:288] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28fb493e03bf4b0ebb31fab310774288 (127.5.61.2:40727), d180be1f19c5447c8571bab587f3b515 (127.5.61.3:42109)
I20260809 20:22:09.416137 26159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:09.409236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:38519 (local address 127.5.61.3:42109)
0809 20:22:09.409473 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:09.409481 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:09.409814 (+ 333us) server_negotiation.cc:407] Connection header received
0809 20:22:09.409990 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:09.409994 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:09.410050 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:09.410123 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:09.410838 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.411885 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.413856 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.415159 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.415302 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:09.415414 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:09.415730 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 20:22:09.415843 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 20:22:09.416239 26172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:09.409064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:42109 (local address 127.5.61.1:38519)
0809 20:22:09.409694 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:09.409718 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:09.409844 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:09.410221 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:09.410227 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:09.410257 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:09.410620 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:09.410632 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.412022 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.412027 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:09.413634 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:09.413651 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.415332 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.415336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:09.415448 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:09.415472 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 20:22:09.415678 (+ 206us) client_negotiation.cc:240] Negotiation successful
0809 20:22:09.415803 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"mutex_wait_us":1,"thread_start_us":284,"threads_started":1}
I20260809 20:22:09.416385 26156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:09.408166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:52403 (local address 127.5.61.2:40727)
0809 20:22:09.408406 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:09.408423 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:09.409042 (+ 619us) server_negotiation.cc:407] Connection header received
0809 20:22:09.409277 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:09.409286 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:09.409410 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:09.409510 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:09.410380 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.411481 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.414105 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.415412 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.415536 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:09.415703 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:09.415967 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 20:22:09.416098 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 20:22:09.416697 26171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:09.408084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:40727 (local address 127.5.61.1:52403)
0809 20:22:09.408895 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:09.408912 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:09.409091 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:09.409576 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:09.409581 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:09.409610 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:09.410225 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:09.410239 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.411752 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.411758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:09.413923 (+ 2165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:09.413936 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.415642 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.415651 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:09.415806 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:09.415837 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 20:22:09.415924 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 20:22:09.416135 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"mutex_wait_us":513,"thread_start_us":513,"threads_started":1}
I20260809 20:22:09.417346 26081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99d31f9201d245fcb30496eea4d20307" candidate_uuid: "a0382dc3216e49caa8e396ab3536a147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d180be1f19c5447c8571bab587f3b515" is_pre_election: true
I20260809 20:22:09.417721 25929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99d31f9201d245fcb30496eea4d20307" candidate_uuid: "a0382dc3216e49caa8e396ab3536a147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28fb493e03bf4b0ebb31fab310774288" is_pre_election: true
I20260809 20:22:09.417977 26081 raft_consensus.cc:2454] T 99d31f9201d245fcb30496eea4d20307 P d180be1f19c5447c8571bab587f3b515 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0382dc3216e49caa8e396ab3536a147 in term 0.
I20260809 20:22:09.418313 25929 raft_consensus.cc:2454] T 99d31f9201d245fcb30496eea4d20307 P 28fb493e03bf4b0ebb31fab310774288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0382dc3216e49caa8e396ab3536a147 in term 0.
I20260809 20:22:09.418766 25699 leader_election.cc:302] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0382dc3216e49caa8e396ab3536a147, d180be1f19c5447c8571bab587f3b515; no voters:
I20260809 20:22:09.419188 26169 raft_consensus.cc:2781] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:22:09.419332 26169 raft_consensus.cc:482] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:22:09.419464 26169 raft_consensus.cc:3037] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:09.421733 26169 raft_consensus.cc:504] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28fb493e03bf4b0ebb31fab310774288" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 40727 } } peers { permanent_uuid: "a0382dc3216e49caa8e396ab3536a147" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 46737 } } peers { permanent_uuid: "d180be1f19c5447c8571bab587f3b515" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 42109 } }
I20260809 20:22:09.422319 26169 leader_election.cc:288] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147 [CANDIDATE]: Term 1 election: Requested vote from peers 28fb493e03bf4b0ebb31fab310774288 (127.5.61.2:40727), d180be1f19c5447c8571bab587f3b515 (127.5.61.3:42109)
I20260809 20:22:09.422823 25929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99d31f9201d245fcb30496eea4d20307" candidate_uuid: "a0382dc3216e49caa8e396ab3536a147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28fb493e03bf4b0ebb31fab310774288"
I20260809 20:22:09.422935 26081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99d31f9201d245fcb30496eea4d20307" candidate_uuid: "a0382dc3216e49caa8e396ab3536a147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d180be1f19c5447c8571bab587f3b515"
I20260809 20:22:09.423028 25929 raft_consensus.cc:3037] T 99d31f9201d245fcb30496eea4d20307 P 28fb493e03bf4b0ebb31fab310774288 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:09.423166 26081 raft_consensus.cc:3037] T 99d31f9201d245fcb30496eea4d20307 P d180be1f19c5447c8571bab587f3b515 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:09.425328 25929 raft_consensus.cc:2454] T 99d31f9201d245fcb30496eea4d20307 P 28fb493e03bf4b0ebb31fab310774288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0382dc3216e49caa8e396ab3536a147 in term 1.
I20260809 20:22:09.425451 26081 raft_consensus.cc:2454] T 99d31f9201d245fcb30496eea4d20307 P d180be1f19c5447c8571bab587f3b515 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0382dc3216e49caa8e396ab3536a147 in term 1.
I20260809 20:22:09.425698 25699 leader_election.cc:302] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28fb493e03bf4b0ebb31fab310774288, a0382dc3216e49caa8e396ab3536a147; no voters:
I20260809 20:22:09.425985 26169 raft_consensus.cc:2781] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:09.426441 26169 raft_consensus.cc:686] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147 [term 1 LEADER]: Becoming Leader. State: Replica: a0382dc3216e49caa8e396ab3536a147, State: Running, Role: LEADER
I20260809 20:22:09.426896 26169 consensus_queue.cc:237] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28fb493e03bf4b0ebb31fab310774288" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 40727 } } peers { permanent_uuid: "a0382dc3216e49caa8e396ab3536a147" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 46737 } } peers { permanent_uuid: "d180be1f19c5447c8571bab587f3b515" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 42109 } }
I20260809 20:22:09.431918 25604 catalog_manager.cc:5495] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0382dc3216e49caa8e396ab3536a147 (127.5.61.1). New cstate: current_term: 1 leader_uuid: "a0382dc3216e49caa8e396ab3536a147" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28fb493e03bf4b0ebb31fab310774288" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 40727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0382dc3216e49caa8e396ab3536a147" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 46737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d180be1f19c5447c8571bab587f3b515" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 42109 } health_report { overall_health: UNKNOWN } } }
W20260809 20:22:09.435945 26141 tablet.cc:2369] T 99d31f9201d245fcb30496eea4d20307 P d180be1f19c5447c8571bab587f3b515: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:22:09.460151 26158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:09.455057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51078 (local address 127.5.61.1:46737)
0809 20:22:09.455268 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:09.455273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:09.455290 (+ 17us) server_negotiation.cc:407] Connection header received
0809 20:22:09.455456 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:09.455460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:09.455546 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:09.455715 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:22:09.456029 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.456710 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.458052 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.458532 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.458622 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:09.459058 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:22:09.459592 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:22:09.459715 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:09.459809 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 20:22:09.459870 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 20:22:09.468451 26081 raft_consensus.cc:1261] T 99d31f9201d245fcb30496eea4d20307 P d180be1f19c5447c8571bab587f3b515 [term 1 FOLLOWER]: Refusing update from remote peer a0382dc3216e49caa8e396ab3536a147: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:22:09.468530 25929 raft_consensus.cc:1261] T 99d31f9201d245fcb30496eea4d20307 P 28fb493e03bf4b0ebb31fab310774288 [term 1 FOLLOWER]: Refusing update from remote peer a0382dc3216e49caa8e396ab3536a147: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:22:09.469406 26173 consensus_queue.cc:1035] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d180be1f19c5447c8571bab587f3b515" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 42109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:22:09.469774 26169 consensus_queue.cc:1035] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28fb493e03bf4b0ebb31fab310774288" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 40727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 20:22:09.475319 25983 tablet.cc:2369] T 99d31f9201d245fcb30496eea4d20307 P 28fb493e03bf4b0ebb31fab310774288: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:22:09.479658 26176 mvcc.cc:205] Tried to move back new op lower bound from 7316713183091916800 to 7316713182936576000. Current Snapshot: MvccSnapshot[applied={T|T < 7316713183091916800}]
I20260809 20:22:09.482446 26178 mvcc.cc:205] Tried to move back new op lower bound from 7316713183091916800 to 7316713182936576000. Current Snapshot: MvccSnapshot[applied={T|T < 7316713183091916800}]
I20260809 20:22:09.499802 25606 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:22:09.500062 25606 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:22:09.503746 25606 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b94ae85dbec64d70ac73b36d25a2fded: Sending DeleteTablet for 3 replicas of tablet 99d31f9201d245fcb30496eea4d20307
I20260809 20:22:09.504922 25754 tablet_service.cc:1514] Processing DeleteTablet for tablet 99d31f9201d245fcb30496eea4d20307 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:22:09 UTC) from {username='slave'} at 127.0.0.1:51064
I20260809 20:22:09.504900 25905 tablet_service.cc:1514] Processing DeleteTablet for tablet 99d31f9201d245fcb30496eea4d20307 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:22:09 UTC) from {username='slave'} at 127.0.0.1:40870
I20260809 20:22:09.505782 26186 tablet_replica.cc:330] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147: stopping tablet replica
I20260809 20:22:09.506476 26187 tablet_replica.cc:330] T 99d31f9201d245fcb30496eea4d20307 P 28fb493e03bf4b0ebb31fab310774288: stopping tablet replica
I20260809 20:22:09.506464 26058 tablet_service.cc:1514] Processing DeleteTablet for tablet 99d31f9201d245fcb30496eea4d20307 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:22:09 UTC) from {username='slave'} at 127.0.0.1:51030
I20260809 20:22:09.506853 26186 raft_consensus.cc:2229] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147 [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:22:09.506855 26187 raft_consensus.cc:2229] T 99d31f9201d245fcb30496eea4d20307 P 28fb493e03bf4b0ebb31fab310774288 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:22:09.506897 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 25677
I20260809 20:22:09.507336 26187 raft_consensus.cc:2258] T 99d31f9201d245fcb30496eea4d20307 P 28fb493e03bf4b0ebb31fab310774288 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:22:09.507462 26186 raft_consensus.cc:2258] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:22:09.508919 26188 tablet_replica.cc:330] T 99d31f9201d245fcb30496eea4d20307 P d180be1f19c5447c8571bab587f3b515: stopping tablet replica
I20260809 20:22:09.509249 26188 raft_consensus.cc:2229] T 99d31f9201d245fcb30496eea4d20307 P d180be1f19c5447c8571bab587f3b515 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:22:09.509627 26188 raft_consensus.cc:2258] T 99d31f9201d245fcb30496eea4d20307 P d180be1f19c5447c8571bab587f3b515 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:22:09.510705 26186 ts_tablet_manager.cc:1905] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:22:09.511579 26188 ts_tablet_manager.cc:1905] T 99d31f9201d245fcb30496eea4d20307 P d180be1f19c5447c8571bab587f3b515: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:22:09.514257 26187 ts_tablet_manager.cc:1905] T 99d31f9201d245fcb30496eea4d20307 P 28fb493e03bf4b0ebb31fab310774288: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:22:09.518741 26188 ts_tablet_manager.cc:1918] T 99d31f9201d245fcb30496eea4d20307 P d180be1f19c5447c8571bab587f3b515: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:22:09.518961 26188 log.cc:1196] T 99d31f9201d245fcb30496eea4d20307 P d180be1f19c5447c8571bab587f3b515: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/99d31f9201d245fcb30496eea4d20307
I20260809 20:22:09.519490 26186 ts_tablet_manager.cc:1918] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:22:09.519626 26188 ts_tablet_manager.cc:1939] T 99d31f9201d245fcb30496eea4d20307 P d180be1f19c5447c8571bab587f3b515: Deleting consensus metadata
I20260809 20:22:09.519801 26186 log.cc:1196] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/99d31f9201d245fcb30496eea4d20307
I20260809 20:22:09.520491 26186 ts_tablet_manager.cc:1939] T 99d31f9201d245fcb30496eea4d20307 P a0382dc3216e49caa8e396ab3536a147: Deleting consensus metadata
I20260809 20:22:09.521448 25589 catalog_manager.cc:4841] TS d180be1f19c5447c8571bab587f3b515 (127.5.61.3:42109): tablet 99d31f9201d245fcb30496eea4d20307 (table test-table [id=8836e98631214f64a6101b732f5feafc]) successfully deleted
I20260809 20:22:09.522217 25591 catalog_manager.cc:4841] TS a0382dc3216e49caa8e396ab3536a147 (127.5.61.1:46737): tablet 99d31f9201d245fcb30496eea4d20307 (table test-table [id=8836e98631214f64a6101b732f5feafc]) successfully deleted
I20260809 20:22:09.526291 26158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:09.508085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51088 (local address 127.5.61.1:46737)
0809 20:22:09.508335 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:09.508352 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:09.514818 (+ 6466us) server_negotiation.cc:407] Connection header received
0809 20:22:09.514989 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:09.514994 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:09.515093 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:09.515219 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:09.516093 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.517554 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.522170 (+ 4616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.522877 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.523007 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:09.524676 (+ 1669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:09.524687 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:09.524702 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:09.525412 (+ 710us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:09.525466 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:09.525495 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:09.525603 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:09.525805 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 20:22:09.525921 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260809 20:22:09.526695 26187 ts_tablet_manager.cc:1918] T 99d31f9201d245fcb30496eea4d20307 P 28fb493e03bf4b0ebb31fab310774288: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:22:09.526889 26187 log.cc:1196] T 99d31f9201d245fcb30496eea4d20307 P 28fb493e03bf4b0ebb31fab310774288: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/99d31f9201d245fcb30496eea4d20307
I20260809 20:22:09.527038 25823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:09.527621 26187 ts_tablet_manager.cc:1939] T 99d31f9201d245fcb30496eea4d20307 P 28fb493e03bf4b0ebb31fab310774288: Deleting consensus metadata
I20260809 20:22:09.530143 25589 catalog_manager.cc:4841] TS 28fb493e03bf4b0ebb31fab310774288 (127.5.61.2:40727): tablet 99d31f9201d245fcb30496eea4d20307 (table test-table [id=8836e98631214f64a6101b732f5feafc]) successfully deleted
I20260809 20:22:09.623379 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 25677
I20260809 20:22:09.643769 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 25834
I20260809 20:22:09.655776 26156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:09.644550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40882 (local address 127.5.61.2:40727)
0809 20:22:09.644721 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:09.644725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:09.644755 (+ 30us) server_negotiation.cc:407] Connection header received
0809 20:22:09.644864 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:09.644867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:09.644968 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:09.645054 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:09.645436 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.646200 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.647408 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.647850 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.647939 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:09.654467 (+ 6528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:09.654475 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:09.654489 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:09.655085 (+ 596us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:09.655138 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:09.655157 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:09.655250 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:09.655351 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 20:22:09.655441 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 20:22:09.656463 25977 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:09.744867 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 25834
I20260809 20:22:09.767143 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 25986
I20260809 20:22:09.774149 26159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:09.768019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51038 (local address 127.5.61.3:42109)
0809 20:22:09.768212 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:09.768219 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:09.768240 (+ 21us) server_negotiation.cc:407] Connection header received
0809 20:22:09.768441 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:09.768447 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:09.768542 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:09.768650 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:09.769063 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.769842 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.771153 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:09.771632 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:09.771762 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:09.772822 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:09.772831 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:09.772845 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:09.773417 (+ 572us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:09.773464 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:09.773483 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:09.773607 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:09.773731 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 20:22:09.773839 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 20:22:09.774811 26133 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:09.861351 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 25986
I20260809 20:22:09.882354 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 25571
I20260809 20:22:09.883502 25641 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:09.992054 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 25571
2026-08-09T20:22:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2855 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:22:10 dist-test-slave-6k22 krb5kdc[26196](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:22:10 dist-test-slave-6k22 krb5kdc[26196](info): set up 2 sockets
Aug 09 20:22:10 dist-test-slave-6k22 krb5kdc[26196](info): commencing operation
krb5kdc: starting...
W20260809 20:22:11.201056 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.002s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:22:11 dist-test-slave-6k22 krb5kdc[26196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:22:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:22:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:22:11.502317 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:40143
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:43919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:40143
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:11.839138 26210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:11.839461 26210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:11.839687 26210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:11.846664 26210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:22:11.846796 26210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:11.846904 26210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:11.847008 26210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:22:11.847096 26210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:22:11.856531 26210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:43919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:40143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:40143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:11.857831 26210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:11.859232 26210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:11.871078 26216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:11.871037 26217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:11.871321 26219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:11.871984 26210 server_base.cc:1034] running on GCE node
I20260809 20:22:11.872738 26210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:11.874073 26210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:11.875315 26210 hybrid_clock.cc:648] HybridClock initialized: now 1786306931875274 us; error 51 us; skew 500 ppm
I20260809 20:22:11.875712 26210 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:11 dist-test-slave-6k22 krb5kdc[26196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306931, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:11.878746 26210 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:22:11.880198 26210 webserver.cc:460] Webserver started at http://127.5.61.62:37195/ using document root <none> and password file <none>
I20260809 20:22:11.880837 26210 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:11.880934 26210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:11.881170 26210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:11.884027 26210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "bf456e98bf9c4d38937397e61467cc7b"
format_stamp: "Formatted at 2026-08-09 20:22:11 on dist-test-slave-6k22"
server_key: "12c5b688cf5b57f7ba16c7fce19fdc8d"
server_key_iv: "20edbc72496147fcaefdbea2f1097fba"
server_key_version: "encryptionkey@0"
I20260809 20:22:11.884727 26210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "bf456e98bf9c4d38937397e61467cc7b"
format_stamp: "Formatted at 2026-08-09 20:22:11 on dist-test-slave-6k22"
server_key: "12c5b688cf5b57f7ba16c7fce19fdc8d"
server_key_iv: "20edbc72496147fcaefdbea2f1097fba"
server_key_version: "encryptionkey@0"
I20260809 20:22:11.889613 26210 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 20:22:11.892982 26230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:11.894307 26210 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:22:11.894446 26210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "bf456e98bf9c4d38937397e61467cc7b"
format_stamp: "Formatted at 2026-08-09 20:22:11 on dist-test-slave-6k22"
server_key: "12c5b688cf5b57f7ba16c7fce19fdc8d"
server_key_iv: "20edbc72496147fcaefdbea2f1097fba"
server_key_version: "encryptionkey@0"
I20260809 20:22:11.894598 26210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:11.909420 26210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:11.913317 26210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:11.913540 26210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:11.931845 26210 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:40143
I20260809 20:22:11.931950 26299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:40143 every 8 connection(s)
I20260809 20:22:11.933281 26210 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:22:11.936972 26300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:11.939949 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 26210
I20260809 20:22:11.940141 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:22:11.940572 5364 external_mini_cluster.cc:1428] Setting key 38ef9ca2e5717ddd903cedd6cbb5f6a7
I20260809 20:22:11.948916 26300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf456e98bf9c4d38937397e61467cc7b: Bootstrap starting.
I20260809 20:22:11.952527 26300 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf456e98bf9c4d38937397e61467cc7b: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:11.953816 26300 log.cc:824] T 00000000000000000000000000000000 P bf456e98bf9c4d38937397e61467cc7b: Log is configured to *not* fsync() on all Append() calls
Aug 09 20:22:11 dist-test-slave-6k22 krb5kdc[26196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:22:11.956699 26300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf456e98bf9c4d38937397e61467cc7b: No bootstrap required, opened a new log
I20260809 20:22:11.964811 26303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:11.942301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49452 (local address 127.5.61.62:40143)
0809 20:22:11.943061 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:11.943072 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:11.943112 (+ 40us) server_negotiation.cc:407] Connection header received
0809 20:22:11.944831 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:11.944863 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:11.945253 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:11.945582 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:11.947507 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:11.948807 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:11.950623 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:11.951250 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:11.951371 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:11.956378 (+ 5007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:11.956388 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:11.956405 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:11.956446 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:11.960426 (+ 3980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:11.961058 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:11.961067 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:11.961091 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:11.961319 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:11.961672 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:11.961677 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:11.961698 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:11.962378 (+ 680us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:11.962691 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:11.962992 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 20:22:11.963444 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":366,"threads_started":1}
I20260809 20:22:11.967617 26300 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf456e98bf9c4d38937397e61467cc7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf456e98bf9c4d38937397e61467cc7b" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40143 } }
I20260809 20:22:11.967986 26300 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf456e98bf9c4d38937397e61467cc7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:11.968055 26300 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf456e98bf9c4d38937397e61467cc7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf456e98bf9c4d38937397e61467cc7b, State: Initialized, Role: FOLLOWER
I20260809 20:22:11.968484 26300 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf456e98bf9c4d38937397e61467cc7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf456e98bf9c4d38937397e61467cc7b" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40143 } }
I20260809 20:22:11.968695 26300 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf456e98bf9c4d38937397e61467cc7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:22:11.968842 26300 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf456e98bf9c4d38937397e61467cc7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:22:11.968994 26300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf456e98bf9c4d38937397e61467cc7b [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:11.971035 26300 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf456e98bf9c4d38937397e61467cc7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf456e98bf9c4d38937397e61467cc7b" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40143 } }
I20260809 20:22:11.971449 26300 leader_election.cc:302] T 00000000000000000000000000000000 P bf456e98bf9c4d38937397e61467cc7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf456e98bf9c4d38937397e61467cc7b; no voters:
I20260809 20:22:11.972110 26300 leader_election.cc:288] T 00000000000000000000000000000000 P bf456e98bf9c4d38937397e61467cc7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:22:11.972270 26305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf456e98bf9c4d38937397e61467cc7b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:11.973465 26305 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf456e98bf9c4d38937397e61467cc7b [term 1 LEADER]: Becoming Leader. State: Replica: bf456e98bf9c4d38937397e61467cc7b, State: Running, Role: LEADER
I20260809 20:22:11.974009 26305 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf456e98bf9c4d38937397e61467cc7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf456e98bf9c4d38937397e61467cc7b" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40143 } }
I20260809 20:22:11.974519 26300 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf456e98bf9c4d38937397e61467cc7b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:22:11.976847 26306 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf456e98bf9c4d38937397e61467cc7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf456e98bf9c4d38937397e61467cc7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf456e98bf9c4d38937397e61467cc7b" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40143 } } }
I20260809 20:22:11.977295 26306 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf456e98bf9c4d38937397e61467cc7b [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:11.977626 26307 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf456e98bf9c4d38937397e61467cc7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf456e98bf9c4d38937397e61467cc7b. Latest consensus state: current_term: 1 leader_uuid: "bf456e98bf9c4d38937397e61467cc7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf456e98bf9c4d38937397e61467cc7b" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40143 } } }
I20260809 20:22:11.977963 26307 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf456e98bf9c4d38937397e61467cc7b [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:11.980820 26317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:22:11.985126 26317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:22:11.993031 26317 catalog_manager.cc:1347] Generated new cluster ID: f9db5fd09d9147c3967f1b7a083627f1
I20260809 20:22:11.993168 26317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:22:12.008221 26317 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:22:12.010030 26317 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:22:12.024979 26317 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf456e98bf9c4d38937397e61467cc7b: Generated new TSK 0
I20260809 20:22:12.025991 26317 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.61.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
I20260809 20:22:12.099093 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:40143
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:43919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:12.445717 26330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:12.445991 26330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:12.446202 26330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:12.453223 26330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:12.453333 26330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:12.453563 26330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:22:12.463052 26330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:43919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:40143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:12.464540 26330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:12.465845 26330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:12.479681 26337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:12.479591 26341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:12.479640 26336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:12.480846 26330 server_base.cc:1034] running on GCE node
I20260809 20:22:12.481419 26330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:12.482296 26330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:12.483497 26330 hybrid_clock.cc:648] HybridClock initialized: now 1786306932483459 us; error 39 us; skew 500 ppm
I20260809 20:22:12.483846 26330 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:12 dist-test-slave-6k22 krb5kdc[26196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306932, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:12.487017 26330 init.cc:377] Logged in from keytab as kudu/127.5.61.1@KRBTEST.COM (short username kudu)
I20260809 20:22:12.488440 26330 webserver.cc:460] Webserver started at http://127.5.61.1:42245/ using document root <none> and password file <none>
I20260809 20:22:12.489105 26330 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:12.489233 26330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:12.489545 26330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:12.492379 26330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "3a663f4e52a2484b9e08bf94c8078ce5"
format_stamp: "Formatted at 2026-08-09 20:22:12 on dist-test-slave-6k22"
server_key: "40f20160e968b8559fb876307422f41e"
server_key_iv: "533f4f23200180b2b1320505f1064613"
server_key_version: "encryptionkey@0"
I20260809 20:22:12.493201 26330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "3a663f4e52a2484b9e08bf94c8078ce5"
format_stamp: "Formatted at 2026-08-09 20:22:12 on dist-test-slave-6k22"
server_key: "40f20160e968b8559fb876307422f41e"
server_key_iv: "533f4f23200180b2b1320505f1064613"
server_key_version: "encryptionkey@0"
I20260809 20:22:12.498492 26330 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 20:22:12.501876 26348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:12.503839 26330 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:22:12.504004 26330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "3a663f4e52a2484b9e08bf94c8078ce5"
format_stamp: "Formatted at 2026-08-09 20:22:12 on dist-test-slave-6k22"
server_key: "40f20160e968b8559fb876307422f41e"
server_key_iv: "533f4f23200180b2b1320505f1064613"
server_key_version: "encryptionkey@0"
I20260809 20:22:12.504200 26330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:12.544927 26330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:12.549329 26330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:12.549654 26330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:12.550593 26330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:22:12.551968 26330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:22:12.552040 26330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:12.552197 26330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:22:12.552255 26330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:12.587304 26330 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:37105
I20260809 20:22:12.587381 26478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:37105 every 8 connection(s)
I20260809 20:22:12.588694 26330 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:22:12.589247 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 26330
I20260809 20:22:12.589464 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:22:12.590087 5364 external_mini_cluster.cc:1428] Setting key 6ad82b4ac342927fb5925c1a5e08de34
Aug 09 20:22:12 dist-test-slave-6k22 krb5kdc[26196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306932, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:22:12.611486 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:12.591718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:45593 (local address 127.5.61.62:40143)
0809 20:22:12.592183 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:12.592189 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:12.593462 (+ 1273us) server_negotiation.cc:407] Connection header received
0809 20:22:12.596353 (+ 2891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:12.596371 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:12.596435 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:12.596533 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:12.598624 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:12.599331 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:12.600971 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:12.601383 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:12.601521 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:12.605432 (+ 3911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:12.605437 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:12.605440 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:12.605471 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:12.608431 (+ 2960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:12.609173 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:12.609178 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:12.609181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:12.609313 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:12.609753 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:12.609758 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:12.609761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:12.610053 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:12.610209 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:12.610882 (+ 673us) server_negotiation.cc:299] Negotiation successful
0809 20:22:12.611105 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":426,"threads_started":1}
I20260809 20:22:12.612561 26484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:12.592395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40143 (local address 127.5.61.1:45593)
0809 20:22:12.593264 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:12.593331 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:12.596043 (+ 2712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:12.596800 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:12.596814 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:12.597570 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:22:12.598447 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:12.598476 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:12.599547 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:12.599605 (+ 58us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:12.600812 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:12.600824 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:12.601617 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:12.601625 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:12.601847 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:12.602524 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:22:12.602543 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:22:12.605213 (+ 2670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:22:12.608567 (+ 3354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:22:12.608575 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:22:12.608589 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:22:12.609045 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:22:12.609439 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:22:12.609445 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:22:12.609453 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:22:12.609630 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:22:12.610190 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:22:12.610197 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:22:12.610511 (+ 314us) client_negotiation.cc:769] Sending connection context
0809 20:22:12.610760 (+ 249us) client_negotiation.cc:240] Negotiation successful
0809 20:22:12.611293 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":349,"threads_started":1}
I20260809 20:22:12.614638 26479 heartbeater.cc:347] Connected to a master server at 127.5.61.62:40143
I20260809 20:22:12.614974 26479 heartbeater.cc:464] Registering TS with master...
I20260809 20:22:12.615779 26479 heartbeater.cc:511] Master 127.5.61.62:40143 requested a full tablet report, sending...
I20260809 20:22:12.618108 26259 ts_manager.cc:194] Registered new tserver with Master: 3a663f4e52a2484b9e08bf94c8078ce5 (127.5.61.1:37105)
I20260809 20:22:12.620383 26259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.1@KRBTEST.COM'} at 127.5.61.1:45593
WARNING: no policy specified for kudu/127.5.61.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
I20260809 20:22:12.658905 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:40143
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:43919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:12.999019 26487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:12.999300 26487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:12.999547 26487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:13.006752 26487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:13.006880 26487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:13.007050 26487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:22:13.016650 26487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:43919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:40143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:13.017864 26487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:13.019227 26487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:13.032156 26496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:13.032162 26494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:13.032444 26493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:13.033085 26487 server_base.cc:1034] running on GCE node
I20260809 20:22:13.033716 26487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:13.034612 26487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:13.035835 26487 hybrid_clock.cc:648] HybridClock initialized: now 1786306933035785 us; error 59 us; skew 500 ppm
I20260809 20:22:13.036163 26487 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:13 dist-test-slave-6k22 krb5kdc[26196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306933, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:13.039268 26487 init.cc:377] Logged in from keytab as kudu/127.5.61.2@KRBTEST.COM (short username kudu)
I20260809 20:22:13.040645 26487 webserver.cc:460] Webserver started at http://127.5.61.2:35153/ using document root <none> and password file <none>
I20260809 20:22:13.041299 26487 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:13.041445 26487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:13.041752 26487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:13.044716 26487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "389aef31a1554372b182181b63e5da40"
format_stamp: "Formatted at 2026-08-09 20:22:13 on dist-test-slave-6k22"
server_key: "be4d8465ec6a2ad61330717fdc3c29cf"
server_key_iv: "f91485fe337bf4cba0a7042a10706090"
server_key_version: "encryptionkey@0"
I20260809 20:22:13.045480 26487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "389aef31a1554372b182181b63e5da40"
format_stamp: "Formatted at 2026-08-09 20:22:13 on dist-test-slave-6k22"
server_key: "be4d8465ec6a2ad61330717fdc3c29cf"
server_key_iv: "f91485fe337bf4cba0a7042a10706090"
server_key_version: "encryptionkey@0"
I20260809 20:22:13.050422 26487 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 20:22:13.053663 26503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:13.054852 26487 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 20:22:13.055014 26487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "389aef31a1554372b182181b63e5da40"
format_stamp: "Formatted at 2026-08-09 20:22:13 on dist-test-slave-6k22"
server_key: "be4d8465ec6a2ad61330717fdc3c29cf"
server_key_iv: "f91485fe337bf4cba0a7042a10706090"
server_key_version: "encryptionkey@0"
I20260809 20:22:13.055197 26487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:13.083997 26487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:13.087994 26487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:13.088289 26487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:13.089290 26487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:22:13.090598 26487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:22:13.090694 26487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:13.090832 26487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:22:13.090909 26487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:13.124503 26487 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:41697
I20260809 20:22:13.124603 26639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:41697 every 8 connection(s)
I20260809 20:22:13.125821 26487 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:22:13.128535 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 26487
I20260809 20:22:13.128724 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:22:13.129273 5364 external_mini_cluster.cc:1428] Setting key 9467ae4fc64000fc391a5b55f61603e5
Aug 09 20:22:13 dist-test-slave-6k22 krb5kdc[26196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306933, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:22:13.148145 26642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:13.128429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:50691 (local address 127.5.61.62:40143)
0809 20:22:13.128908 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:13.128913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:13.130057 (+ 1144us) server_negotiation.cc:407] Connection header received
0809 20:22:13.132189 (+ 2132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:13.132197 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:13.132286 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:13.132413 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:13.133872 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:13.135059 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:13.137227 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:13.137960 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:13.138101 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:13.141494 (+ 3393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:13.141502 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:13.141509 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:13.141565 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:13.144858 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:13.145810 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:13.145814 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:13.145816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:13.145955 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:13.146449 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:13.146451 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:13.146453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:13.146692 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:13.146820 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:13.147701 (+ 881us) server_negotiation.cc:299] Negotiation successful
0809 20:22:13.147875 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":256,"threads_started":1}
I20260809 20:22:13.149636 26643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:13.129067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40143 (local address 127.5.61.2:50691)
0809 20:22:13.129849 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:13.129921 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:13.131923 (+ 2002us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:13.132616 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:13.132623 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:13.133143 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:22:13.133680 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:13.133697 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:13.135219 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:13.135226 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:13.137092 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:13.137104 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:13.138149 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:13.138153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:13.138337 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:13.139046 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:22:13.139067 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:22:13.141269 (+ 2202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:22:13.144992 (+ 3723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:22:13.145002 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:22:13.145018 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:22:13.145681 (+ 663us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:22:13.146060 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:22:13.146067 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:22:13.146074 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:22:13.146331 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:22:13.146834 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:22:13.146844 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:22:13.147271 (+ 427us) client_negotiation.cc:769] Sending connection context
0809 20:22:13.147592 (+ 321us) client_negotiation.cc:240] Negotiation successful
0809 20:22:13.148159 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":358,"threads_started":1}
I20260809 20:22:13.151093 26640 heartbeater.cc:347] Connected to a master server at 127.5.61.62:40143
I20260809 20:22:13.151387 26640 heartbeater.cc:464] Registering TS with master...
I20260809 20:22:13.152137 26640 heartbeater.cc:511] Master 127.5.61.62:40143 requested a full tablet report, sending...
I20260809 20:22:13.153579 26259 ts_manager.cc:194] Registered new tserver with Master: 389aef31a1554372b182181b63e5da40 (127.5.61.2:41697)
I20260809 20:22:13.154822 26259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.2@KRBTEST.COM'} at 127.5.61.2:50691
WARNING: no policy specified for kudu/127.5.61.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
I20260809 20:22:13.198529 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:40143
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:43919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:13.534940 26647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:13.535197 26647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:13.535386 26647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:13.542879 26647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:13.543004 26647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:13.543114 26647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:22:13.552502 26647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:43919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:40143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:13.553674 26647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:13.554898 26647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:13.567950 26653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:13.568209 26655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:13.568241 26652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:13.568907 26647 server_base.cc:1034] running on GCE node
I20260809 20:22:13.569471 26647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:13.570335 26647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:13.571593 26647 hybrid_clock.cc:648] HybridClock initialized: now 1786306933571522 us; error 44 us; skew 500 ppm
I20260809 20:22:13.571913 26647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:13 dist-test-slave-6k22 krb5kdc[26196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306933, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:13.575066 26647 init.cc:377] Logged in from keytab as kudu/127.5.61.3@KRBTEST.COM (short username kudu)
I20260809 20:22:13.576515 26647 webserver.cc:460] Webserver started at http://127.5.61.3:41581/ using document root <none> and password file <none>
I20260809 20:22:13.577193 26647 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:13.577335 26647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:13.577656 26647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:13.580693 26647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "9b760a97f421417dbd99a3a76c5b996e"
format_stamp: "Formatted at 2026-08-09 20:22:13 on dist-test-slave-6k22"
server_key: "44e8aa68587bcb2f39774270cd20c5f0"
server_key_iv: "70b82b0f46fbe85fd6bf998e64cd54fc"
server_key_version: "encryptionkey@0"
I20260809 20:22:13.581480 26647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "9b760a97f421417dbd99a3a76c5b996e"
format_stamp: "Formatted at 2026-08-09 20:22:13 on dist-test-slave-6k22"
server_key: "44e8aa68587bcb2f39774270cd20c5f0"
server_key_iv: "70b82b0f46fbe85fd6bf998e64cd54fc"
server_key_version: "encryptionkey@0"
I20260809 20:22:13.586532 26647 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 20:22:13.590057 26663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:13.591449 26647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 20:22:13.591634 26647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "9b760a97f421417dbd99a3a76c5b996e"
format_stamp: "Formatted at 2026-08-09 20:22:13 on dist-test-slave-6k22"
server_key: "44e8aa68587bcb2f39774270cd20c5f0"
server_key_iv: "70b82b0f46fbe85fd6bf998e64cd54fc"
server_key_version: "encryptionkey@0"
I20260809 20:22:13.591842 26647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:13.605795 26647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:13.609758 26647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:13.610034 26647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:13.610955 26647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:22:13.612269 26647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:22:13.612371 26647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:13.612514 26647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:22:13.612596 26647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:13.623411 26479 heartbeater.cc:503] Master 127.5.61.62:40143 was elected leader, sending a full tablet report...
I20260809 20:22:13.645259 26647 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:39383
I20260809 20:22:13.645288 26797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:39383 every 8 connection(s)
I20260809 20:22:13.646464 26647 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:22:13.655646 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 26647
I20260809 20:22:13.655839 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:22:13.656411 5364 external_mini_cluster.cc:1428] Setting key 6ec280427251e105135d685ae70aefda
Aug 09 20:22:13 dist-test-slave-6k22 krb5kdc[26196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306933, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:22:13.666460 26801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:13.648882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:38233 (local address 127.5.61.62:40143)
0809 20:22:13.649336 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:13.649342 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:13.650384 (+ 1042us) server_negotiation.cc:407] Connection header received
0809 20:22:13.652202 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:13.652206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:13.652285 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:13.652390 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:13.653773 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:13.654510 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:13.656149 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:13.656755 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:13.656907 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:13.660952 (+ 4045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:13.660957 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:13.660960 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:13.660992 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:13.663380 (+ 2388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:13.664143 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:13.664148 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:13.664154 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:13.664279 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:13.664733 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:13.664737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:13.664739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:13.665012 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:13.665167 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:13.665840 (+ 673us) server_negotiation.cc:299] Negotiation successful
0809 20:22:13.666079 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":204,"threads_started":1}
I20260809 20:22:13.667392 26803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:13.649451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40143 (local address 127.5.61.3:38233)
0809 20:22:13.650197 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:13.650252 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:13.651966 (+ 1714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:13.652568 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:13.652575 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:13.653069 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:22:13.653626 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:13.653644 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:13.654629 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:13.654632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:13.656010 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:13.656022 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:13.657017 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:13.657026 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:13.657310 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:13.658067 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:22:13.658088 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:22:13.660729 (+ 2641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:22:13.663576 (+ 2847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:22:13.663587 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:22:13.663602 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:22:13.664012 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:22:13.664400 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:22:13.664405 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:22:13.664408 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:22:13.664602 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:22:13.665142 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:22:13.665149 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:22:13.665464 (+ 315us) client_negotiation.cc:769] Sending connection context
0809 20:22:13.665731 (+ 267us) client_negotiation.cc:240] Negotiation successful
0809 20:22:13.666230 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":277,"threads_started":1}
I20260809 20:22:13.668744 26798 heartbeater.cc:347] Connected to a master server at 127.5.61.62:40143
I20260809 20:22:13.669075 26798 heartbeater.cc:464] Registering TS with master...
I20260809 20:22:13.669875 26798 heartbeater.cc:511] Master 127.5.61.62:40143 requested a full tablet report, sending...
I20260809 20:22:13.671468 26259 ts_manager.cc:194] Registered new tserver with Master: 9b760a97f421417dbd99a3a76c5b996e (127.5.61.3:39383)
I20260809 20:22:13.672390 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 20:22:13.672461 26259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.3@KRBTEST.COM'} at 127.5.61.3:38233
I20260809 20:22:13.690246 26801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:13.677368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49464 (local address 127.5.61.62:40143)
0809 20:22:13.677563 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:13.677567 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:13.678069 (+ 502us) server_negotiation.cc:407] Connection header received
0809 20:22:13.678255 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:13.678260 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:13.678330 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:13.678433 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:13.679896 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:13.680590 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:13.681901 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:13.682309 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:13.682482 (+ 173us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:13.685154 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:13.685160 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:13.685165 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:13.685207 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:13.687498 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:13.688252 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:13.688257 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:13.688261 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:13.688384 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:13.688844 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:13.688849 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:13.688853 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:13.689134 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:13.689400 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:13.689783 (+ 383us) server_negotiation.cc:299] Negotiation successful
0809 20:22:13.689988 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 20:22:13.693812 26259 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49464:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:22:13.701001 26259 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:22:13.723347 26817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:13.713576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41624 (local address 127.5.61.2:41697)
0809 20:22:13.715122 (+ 1546us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:13.715133 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:13.715160 (+ 27us) server_negotiation.cc:407] Connection header received
0809 20:22:13.715265 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:13.715273 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:13.715541 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:13.716980 (+ 1439us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:13.717232 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:13.718741 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:13.721145 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:13.722078 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:13.722229 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:13.722450 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:13.722848 (+ 398us) server_negotiation.cc:299] Negotiation successful
0809 20:22:13.723026 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1372,"thread_start_us":1249,"threads_started":1}
I20260809 20:22:13.723398 26815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:13.712804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:41697 (local address 127.0.0.1:41624)
0809 20:22:13.713729 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:13.713770 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:13.713944 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:13.716136 (+ 2192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:13.716146 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:13.716195 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:13.716738 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:13.716751 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:13.718875 (+ 2124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:13.718882 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:13.720988 (+ 2106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:13.721000 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:13.722315 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:13.722322 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:13.722558 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:13.722639 (+ 81us) client_negotiation.cc:769] Sending connection context
0809 20:22:13.722777 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 20:22:13.722959 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"mutex_wait_us":5,"thread_start_us":909,"threads_started":1}
I20260809 20:22:13.725157 26820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:13.714605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:39383 (local address 127.0.0.1:60090)
0809 20:22:13.716611 (+ 2006us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:13.716630 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:13.716789 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:13.717416 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:13.717421 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:13.717446 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:13.717881 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:13.717895 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:13.719843 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:13.719848 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:13.722177 (+ 2329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:13.722192 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:13.724239 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:13.724245 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:13.724432 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:13.724475 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 20:22:13.724583 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 20:22:13.724704 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1860,"mutex_wait_us":2,"spinlock_wait_cycles":1792,"thread_start_us":1324,"threads_started":1}
I20260809 20:22:13.725776 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:13.713422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46892 (local address 127.5.61.1:37105)
0809 20:22:13.715582 (+ 2160us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:13.715593 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:13.716673 (+ 1080us) server_negotiation.cc:407] Connection header received
0809 20:22:13.717013 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:13.717021 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:13.717250 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:13.717445 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:13.718445 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:13.719995 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:13.722558 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:13.724300 (+ 1742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:13.724428 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:13.724656 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:13.725261 (+ 605us) server_negotiation.cc:299] Negotiation successful
0809 20:22:13.725445 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1980,"thread_start_us":1817,"threads_started":1}
I20260809 20:22:13.725906 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:13.715151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60090 (local address 127.5.61.3:39383)
0809 20:22:13.715987 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:13.715998 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:13.716922 (+ 924us) server_negotiation.cc:407] Connection header received
0809 20:22:13.717038 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:13.717046 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:13.717235 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:13.717428 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:13.718063 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:13.719638 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:13.722382 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:13.723748 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:13.723878 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:13.724096 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:13.725387 (+ 1291us) server_negotiation.cc:299] Negotiation successful
0809 20:22:13.725537 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":448,"threads_started":1}
I20260809 20:22:13.725718 26818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:13.713192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37105 (local address 127.0.0.1:46892)
0809 20:22:13.716548 (+ 3356us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:13.716572 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:13.716807 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:13.717801 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:13.717805 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:13.717834 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:13.718229 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:13.718241 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:13.720169 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:13.720174 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:13.722404 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:13.722416 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:13.724810 (+ 2394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:13.724815 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:13.725080 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:13.725113 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 20:22:13.725218 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 20:22:13.725352 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3187,"spinlock_wait_cycles":24960,"thread_start_us":1380,"threads_started":1}
I20260809 20:22:13.727855 26562 tablet_service.cc:1467] Processing CreateTablet for tablet cf6c86b5e42340aa82299bce5701edcf (DEFAULT_TABLE table=test-table [id=949658ba28b94fbf8f1fcc5b0a3a665c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:22:13.729179 26562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf6c86b5e42340aa82299bce5701edcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:13.729575 26403 tablet_service.cc:1467] Processing CreateTablet for tablet cf6c86b5e42340aa82299bce5701edcf (DEFAULT_TABLE table=test-table [id=949658ba28b94fbf8f1fcc5b0a3a665c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:22:13.729571 26723 tablet_service.cc:1467] Processing CreateTablet for tablet cf6c86b5e42340aa82299bce5701edcf (DEFAULT_TABLE table=test-table [id=949658ba28b94fbf8f1fcc5b0a3a665c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:22:13.730994 26723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf6c86b5e42340aa82299bce5701edcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:13.730991 26403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf6c86b5e42340aa82299bce5701edcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:13.741163 26821 tablet_bootstrap.cc:492] T cf6c86b5e42340aa82299bce5701edcf P 389aef31a1554372b182181b63e5da40: Bootstrap starting.
I20260809 20:22:13.744889 26821 tablet_bootstrap.cc:654] T cf6c86b5e42340aa82299bce5701edcf P 389aef31a1554372b182181b63e5da40: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:13.745965 26822 tablet_bootstrap.cc:492] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5: Bootstrap starting.
I20260809 20:22:13.746287 26821 log.cc:824] T cf6c86b5e42340aa82299bce5701edcf P 389aef31a1554372b182181b63e5da40: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:13.746894 26823 tablet_bootstrap.cc:492] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e: Bootstrap starting.
I20260809 20:22:13.750159 26822 tablet_bootstrap.cc:654] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:13.750486 26823 tablet_bootstrap.cc:654] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:13.750524 26821 tablet_bootstrap.cc:492] T cf6c86b5e42340aa82299bce5701edcf P 389aef31a1554372b182181b63e5da40: No bootstrap required, opened a new log
I20260809 20:22:13.750795 26821 ts_tablet_manager.cc:1397] T cf6c86b5e42340aa82299bce5701edcf P 389aef31a1554372b182181b63e5da40: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 20:22:13.751325 26822 log.cc:824] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:13.752440 26823 log.cc:824] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:13.755864 26822 tablet_bootstrap.cc:492] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5: No bootstrap required, opened a new log
I20260809 20:22:13.755879 26823 tablet_bootstrap.cc:492] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e: No bootstrap required, opened a new log
I20260809 20:22:13.756150 26822 ts_tablet_manager.cc:1397] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260809 20:22:13.756150 26823 ts_tablet_manager.cc:1397] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 20:22:13.759258 26821 raft_consensus.cc:348] T cf6c86b5e42340aa82299bce5701edcf P 389aef31a1554372b182181b63e5da40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "389aef31a1554372b182181b63e5da40" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 41697 } } peers { permanent_uuid: "3a663f4e52a2484b9e08bf94c8078ce5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37105 } } peers { permanent_uuid: "9b760a97f421417dbd99a3a76c5b996e" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 39383 } }
I20260809 20:22:13.759730 26821 raft_consensus.cc:374] T cf6c86b5e42340aa82299bce5701edcf P 389aef31a1554372b182181b63e5da40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:13.759857 26821 raft_consensus.cc:729] T cf6c86b5e42340aa82299bce5701edcf P 389aef31a1554372b182181b63e5da40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 389aef31a1554372b182181b63e5da40, State: Initialized, Role: FOLLOWER
I20260809 20:22:13.760492 26821 consensus_queue.cc:260] T cf6c86b5e42340aa82299bce5701edcf P 389aef31a1554372b182181b63e5da40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "389aef31a1554372b182181b63e5da40" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 41697 } } peers { permanent_uuid: "3a663f4e52a2484b9e08bf94c8078ce5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37105 } } peers { permanent_uuid: "9b760a97f421417dbd99a3a76c5b996e" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 39383 } }
I20260809 20:22:13.761581 26640 heartbeater.cc:503] Master 127.5.61.62:40143 was elected leader, sending a full tablet report...
I20260809 20:22:13.762347 26821 ts_tablet_manager.cc:1428] T cf6c86b5e42340aa82299bce5701edcf P 389aef31a1554372b182181b63e5da40: Time spent starting tablet: real 0.011s user 0.013s sys 0.000s
I20260809 20:22:13.767783 26822 raft_consensus.cc:348] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "389aef31a1554372b182181b63e5da40" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 41697 } } peers { permanent_uuid: "3a663f4e52a2484b9e08bf94c8078ce5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37105 } } peers { permanent_uuid: "9b760a97f421417dbd99a3a76c5b996e" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 39383 } }
I20260809 20:22:13.767782 26823 raft_consensus.cc:348] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "389aef31a1554372b182181b63e5da40" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 41697 } } peers { permanent_uuid: "3a663f4e52a2484b9e08bf94c8078ce5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37105 } } peers { permanent_uuid: "9b760a97f421417dbd99a3a76c5b996e" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 39383 } }
I20260809 20:22:13.768184 26823 raft_consensus.cc:374] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:13.768184 26822 raft_consensus.cc:374] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:13.768285 26823 raft_consensus.cc:729] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b760a97f421417dbd99a3a76c5b996e, State: Initialized, Role: FOLLOWER
I20260809 20:22:13.768291 26822 raft_consensus.cc:729] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a663f4e52a2484b9e08bf94c8078ce5, State: Initialized, Role: FOLLOWER
I20260809 20:22:13.768889 26823 consensus_queue.cc:260] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "389aef31a1554372b182181b63e5da40" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 41697 } } peers { permanent_uuid: "3a663f4e52a2484b9e08bf94c8078ce5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37105 } } peers { permanent_uuid: "9b760a97f421417dbd99a3a76c5b996e" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 39383 } }
I20260809 20:22:13.768879 26822 consensus_queue.cc:260] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "389aef31a1554372b182181b63e5da40" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 41697 } } peers { permanent_uuid: "3a663f4e52a2484b9e08bf94c8078ce5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37105 } } peers { permanent_uuid: "9b760a97f421417dbd99a3a76c5b996e" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 39383 } }
I20260809 20:22:13.770401 26798 heartbeater.cc:503] Master 127.5.61.62:40143 was elected leader, sending a full tablet report...
I20260809 20:22:13.770704 26822 ts_tablet_manager.cc:1428] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260809 20:22:13.772182 26823 ts_tablet_manager.cc:1428] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
W20260809 20:22:13.841385 26481 tablet.cc:2369] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 20:22:13.877950 26641 tablet.cc:2369] T cf6c86b5e42340aa82299bce5701edcf P 389aef31a1554372b182181b63e5da40: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 20:22:13.898177 26799 tablet.cc:2369] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:22:14.082978 26829 raft_consensus.cc:482] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:22:14.083300 26829 raft_consensus.cc:504] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "389aef31a1554372b182181b63e5da40" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 41697 } } peers { permanent_uuid: "3a663f4e52a2484b9e08bf94c8078ce5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37105 } } peers { permanent_uuid: "9b760a97f421417dbd99a3a76c5b996e" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 39383 } }
I20260809 20:22:14.085115 26829 leader_election.cc:288] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 389aef31a1554372b182181b63e5da40 (127.5.61.2:41697), 3a663f4e52a2484b9e08bf94c8078ce5 (127.5.61.1:37105)
I20260809 20:22:14.091960 26817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:14.085397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:59853 (local address 127.5.61.2:41697)
0809 20:22:14.085605 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:14.085611 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:14.085808 (+ 197us) server_negotiation.cc:407] Connection header received
0809 20:22:14.086028 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:14.086033 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:14.086117 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:14.086227 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:14.086663 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:14.087712 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:14.090030 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:14.091159 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:14.091295 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:14.091405 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:14.091615 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 20:22:14.091744 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 20:22:14.092042 26803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:14.085352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:41697 (local address 127.5.61.3:59853)
0809 20:22:14.085691 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:14.085715 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:14.085849 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:14.086265 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:14.086271 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:14.086306 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:14.086523 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:14.086536 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:14.087833 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:14.087837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:14.089867 (+ 2030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:14.089901 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:14.091306 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:14.091310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:14.091436 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:14.091460 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 20:22:14.091532 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 20:22:14.091719 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":33}
I20260809 20:22:14.092825 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:14.085616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:36421 (local address 127.5.61.1:37105)
0809 20:22:14.085824 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:14.085829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:14.086398 (+ 569us) server_negotiation.cc:407] Connection header received
0809 20:22:14.086540 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:14.086543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:14.086622 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:14.086716 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:14.087157 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:14.088091 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:14.090331 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:14.091491 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:14.091682 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:14.091773 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:14.092174 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 20:22:14.092577 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 20:22:14.093106 26831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:14.085499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37105 (local address 127.5.61.3:36421)
0809 20:22:14.086293 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:14.086308 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:14.086422 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:14.086743 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:14.086745 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:14.086765 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:14.087015 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:14.087022 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:14.088216 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:14.088224 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:14.090194 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:14.090204 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:14.091761 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:14.091766 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:14.091936 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:14.091965 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 20:22:14.092646 (+ 681us) client_negotiation.cc:240] Negotiation successful
0809 20:22:14.092761 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"mutex_wait_us":3,"thread_start_us":476,"threads_started":1}
I20260809 20:22:14.093037 26583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf6c86b5e42340aa82299bce5701edcf" candidate_uuid: "9b760a97f421417dbd99a3a76c5b996e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "389aef31a1554372b182181b63e5da40" is_pre_election: true
I20260809 20:22:14.093662 26583 raft_consensus.cc:2454] T cf6c86b5e42340aa82299bce5701edcf P 389aef31a1554372b182181b63e5da40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b760a97f421417dbd99a3a76c5b996e in term 0.
I20260809 20:22:14.094151 26426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf6c86b5e42340aa82299bce5701edcf" candidate_uuid: "9b760a97f421417dbd99a3a76c5b996e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a663f4e52a2484b9e08bf94c8078ce5" is_pre_election: true
I20260809 20:22:14.094383 26665 leader_election.cc:302] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 389aef31a1554372b182181b63e5da40, 9b760a97f421417dbd99a3a76c5b996e; no voters:
I20260809 20:22:14.094679 26426 raft_consensus.cc:2454] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b760a97f421417dbd99a3a76c5b996e in term 0.
I20260809 20:22:14.094918 26830 raft_consensus.cc:482] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:22:14.095155 26829 raft_consensus.cc:2781] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:22:14.095319 26829 raft_consensus.cc:482] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:22:14.095140 26830 raft_consensus.cc:504] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "389aef31a1554372b182181b63e5da40" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 41697 } } peers { permanent_uuid: "3a663f4e52a2484b9e08bf94c8078ce5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37105 } } peers { permanent_uuid: "9b760a97f421417dbd99a3a76c5b996e" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 39383 } }
I20260809 20:22:14.095458 26829 raft_consensus.cc:3037] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:14.096715 26830 leader_election.cc:288] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 389aef31a1554372b182181b63e5da40 (127.5.61.2:41697), 9b760a97f421417dbd99a3a76c5b996e (127.5.61.3:39383)
I20260809 20:22:14.099077 26829 raft_consensus.cc:504] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "389aef31a1554372b182181b63e5da40" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 41697 } } peers { permanent_uuid: "3a663f4e52a2484b9e08bf94c8078ce5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37105 } } peers { permanent_uuid: "9b760a97f421417dbd99a3a76c5b996e" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 39383 } }
I20260809 20:22:14.100171 26829 leader_election.cc:288] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e [CANDIDATE]: Term 1 election: Requested vote from peers 389aef31a1554372b182181b63e5da40 (127.5.61.2:41697), 3a663f4e52a2484b9e08bf94c8078ce5 (127.5.61.1:37105)
I20260809 20:22:14.100476 26583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf6c86b5e42340aa82299bce5701edcf" candidate_uuid: "9b760a97f421417dbd99a3a76c5b996e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "389aef31a1554372b182181b63e5da40"
I20260809 20:22:14.100704 26583 raft_consensus.cc:3037] T cf6c86b5e42340aa82299bce5701edcf P 389aef31a1554372b182181b63e5da40 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:14.101073 26426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf6c86b5e42340aa82299bce5701edcf" candidate_uuid: "9b760a97f421417dbd99a3a76c5b996e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a663f4e52a2484b9e08bf94c8078ce5"
I20260809 20:22:14.101307 26426 raft_consensus.cc:3037] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:14.104835 26583 raft_consensus.cc:2454] T cf6c86b5e42340aa82299bce5701edcf P 389aef31a1554372b182181b63e5da40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b760a97f421417dbd99a3a76c5b996e in term 1.
I20260809 20:22:14.104959 26426 raft_consensus.cc:2454] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b760a97f421417dbd99a3a76c5b996e in term 1.
I20260809 20:22:14.105340 26665 leader_election.cc:302] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 389aef31a1554372b182181b63e5da40, 9b760a97f421417dbd99a3a76c5b996e; no voters:
I20260809 20:22:14.105803 26829 raft_consensus.cc:2781] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:14.106284 26829 raft_consensus.cc:686] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e [term 1 LEADER]: Becoming Leader. State: Replica: 9b760a97f421417dbd99a3a76c5b996e, State: Running, Role: LEADER
I20260809 20:22:14.106923 26829 consensus_queue.cc:237] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "389aef31a1554372b182181b63e5da40" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 41697 } } peers { permanent_uuid: "3a663f4e52a2484b9e08bf94c8078ce5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37105 } } peers { permanent_uuid: "9b760a97f421417dbd99a3a76c5b996e" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 39383 } }
I20260809 20:22:14.108822 26817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:14.097610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:46385 (local address 127.5.61.2:41697)
0809 20:22:14.097814 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:14.097822 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:14.101472 (+ 3650us) server_negotiation.cc:407] Connection header received
0809 20:22:14.101764 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:14.101772 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:14.101851 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:14.101922 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:14.102851 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:14.104030 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:14.106665 (+ 2635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:14.107807 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:14.107898 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:14.108000 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:14.108434 (+ 434us) server_negotiation.cc:299] Negotiation successful
0809 20:22:14.108562 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 20:22:14.108924 26833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:14.097068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:41697 (local address 127.5.61.1:46385)
0809 20:22:14.101374 (+ 4306us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:14.101391 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:14.101575 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:14.102124 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:14.102128 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:14.102185 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:14.102636 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:14.102650 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:14.104203 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:14.104207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:14.106490 (+ 2283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:14.106505 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:14.108153 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:14.108157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:14.108278 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:14.108298 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 20:22:14.108398 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 20:22:14.108519 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4154,"mutex_wait_us":27,"spinlock_wait_cycles":3456,"thread_start_us":5143,"threads_started":1}
I20260809 20:22:14.109617 26582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf6c86b5e42340aa82299bce5701edcf" candidate_uuid: "3a663f4e52a2484b9e08bf94c8078ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "389aef31a1554372b182181b63e5da40" is_pre_election: true
I20260809 20:22:14.109987 26582 raft_consensus.cc:2379] T cf6c86b5e42340aa82299bce5701edcf P 389aef31a1554372b182181b63e5da40 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a663f4e52a2484b9e08bf94c8078ce5 in current term 1: Already voted for candidate 9b760a97f421417dbd99a3a76c5b996e in this term.
I20260809 20:22:14.110219 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:14.097314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:39761 (local address 127.5.61.3:39383)
0809 20:22:14.097571 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:14.097579 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:14.101100 (+ 3521us) server_negotiation.cc:407] Connection header received
0809 20:22:14.101478 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:14.101486 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:14.101579 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:14.101652 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:14.103041 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:14.104352 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:14.107479 (+ 3127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:14.108933 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:14.109048 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:14.109182 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:14.109853 (+ 671us) server_negotiation.cc:299] Negotiation successful
0809 20:22:14.109986 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 20:22:14.110613 26832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:14.096950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:39383 (local address 127.5.61.1:39761)
0809 20:22:14.100953 (+ 4003us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:14.100980 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:14.101264 (+ 284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:14.101885 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:14.101890 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:14.101923 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:14.102837 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:14.102855 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:14.104600 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:14.104609 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:14.107256 (+ 2647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:14.107272 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:14.109413 (+ 2141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:14.109421 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:14.109632 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:14.109670 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 20:22:14.109803 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 20:22:14.109967 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3803,"mutex_wait_us":287,"thread_start_us":3596,"threads_started":1}
I20260809 20:22:14.111490 26744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf6c86b5e42340aa82299bce5701edcf" candidate_uuid: "3a663f4e52a2484b9e08bf94c8078ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b760a97f421417dbd99a3a76c5b996e" is_pre_election: true
I20260809 20:22:14.112751 26349 leader_election.cc:302] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a663f4e52a2484b9e08bf94c8078ce5; no voters: 389aef31a1554372b182181b63e5da40, 9b760a97f421417dbd99a3a76c5b996e
I20260809 20:22:14.113498 26830 raft_consensus.cc:2726] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 20:22:14.114045 26258 catalog_manager.cc:5495] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b760a97f421417dbd99a3a76c5b996e (127.5.61.3). New cstate: current_term: 1 leader_uuid: "9b760a97f421417dbd99a3a76c5b996e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "389aef31a1554372b182181b63e5da40" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 41697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a663f4e52a2484b9e08bf94c8078ce5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b760a97f421417dbd99a3a76c5b996e" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 39383 } health_report { overall_health: HEALTHY } } }
I20260809 20:22:14.178260 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:14.171786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60094 (local address 127.5.61.3:39383)
0809 20:22:14.171969 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:14.171974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:14.171992 (+ 18us) server_negotiation.cc:407] Connection header received
0809 20:22:14.172169 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:14.172174 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:14.172266 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:14.172385 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:22:14.172816 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:14.173538 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:14.175244 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:14.175794 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:14.175928 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:14.176545 (+ 617us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:22:14.177113 (+ 568us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:22:14.177761 (+ 648us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:14.177856 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 20:22:14.177901 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 20:22:14.186395 26426 raft_consensus.cc:1261] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5 [term 1 FOLLOWER]: Refusing update from remote peer 9b760a97f421417dbd99a3a76c5b996e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:22:14.186420 26582 raft_consensus.cc:1261] T cf6c86b5e42340aa82299bce5701edcf P 389aef31a1554372b182181b63e5da40 [term 1 FOLLOWER]: Refusing update from remote peer 9b760a97f421417dbd99a3a76c5b996e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:22:14.187260 26835 consensus_queue.cc:1035] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a663f4e52a2484b9e08bf94c8078ce5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:22:14.187589 26829 consensus_queue.cc:1035] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e [LEADER]: Connected to new peer: Peer: permanent_uuid: "389aef31a1554372b182181b63e5da40" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 41697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:22:14.198385 26838 mvcc.cc:205] Tried to move back new op lower bound from 7316713202418470912 to 7316713202108264448. Current Snapshot: MvccSnapshot[applied={T|T < 7316713202418470912}]
I20260809 20:22:14.203006 26839 mvcc.cc:205] Tried to move back new op lower bound from 7316713202418470912 to 7316713202108264448. Current Snapshot: MvccSnapshot[applied={T|T < 7316713202418470912}]
I20260809 20:22:14.212790 26258 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:22:14.213114 26258 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:22:14.216809 26258 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bf456e98bf9c4d38937397e61467cc7b: Sending DeleteTablet for 3 replicas of tablet cf6c86b5e42340aa82299bce5701edcf
I20260809 20:22:14.218003 26562 tablet_service.cc:1514] Processing DeleteTablet for tablet cf6c86b5e42340aa82299bce5701edcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:22:14 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:41624
I20260809 20:22:14.217996 26403 tablet_service.cc:1514] Processing DeleteTablet for tablet cf6c86b5e42340aa82299bce5701edcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:22:14 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:46892
I20260809 20:22:14.218706 26723 tablet_service.cc:1514] Processing DeleteTablet for tablet cf6c86b5e42340aa82299bce5701edcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:22:14 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:60090
I20260809 20:22:14.218941 26851 tablet_replica.cc:330] T cf6c86b5e42340aa82299bce5701edcf P 389aef31a1554372b182181b63e5da40: stopping tablet replica
I20260809 20:22:14.219381 26851 raft_consensus.cc:2229] T cf6c86b5e42340aa82299bce5701edcf P 389aef31a1554372b182181b63e5da40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:22:14.219901 26851 raft_consensus.cc:2258] T cf6c86b5e42340aa82299bce5701edcf P 389aef31a1554372b182181b63e5da40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:22:14.222661 26851 ts_tablet_manager.cc:1905] T cf6c86b5e42340aa82299bce5701edcf P 389aef31a1554372b182181b63e5da40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:22:14.223067 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 26330
I20260809 20:22:14.229964 26851 ts_tablet_manager.cc:1918] T cf6c86b5e42340aa82299bce5701edcf P 389aef31a1554372b182181b63e5da40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:22:14.230137 26851 log.cc:1196] T cf6c86b5e42340aa82299bce5701edcf P 389aef31a1554372b182181b63e5da40: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/cf6c86b5e42340aa82299bce5701edcf
I20260809 20:22:14.230748 26851 ts_tablet_manager.cc:1939] T cf6c86b5e42340aa82299bce5701edcf P 389aef31a1554372b182181b63e5da40: Deleting consensus metadata
I20260809 20:22:14.232684 26232 catalog_manager.cc:4841] TS 389aef31a1554372b182181b63e5da40 (127.5.61.2:41697): tablet cf6c86b5e42340aa82299bce5701edcf (table test-table [id=949658ba28b94fbf8f1fcc5b0a3a665c]) successfully deleted
I20260809 20:22:14.233646 26849 tablet_replica.cc:330] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5: stopping tablet replica
I20260809 20:22:14.234174 26849 raft_consensus.cc:2229] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:22:14.234782 26849 raft_consensus.cc:2258] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:22:14.239046 26852 tablet_replica.cc:330] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e: stopping tablet replica
I20260809 20:22:14.239594 26849 ts_tablet_manager.cc:1905] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:22:14.239521 26852 raft_consensus.cc:2229] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:22:14.240351 26852 raft_consensus.cc:2258] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:22:14.242461 26852 ts_tablet_manager.cc:1905] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:22:14.247803 26849 ts_tablet_manager.cc:1918] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:22:14.247972 26849 log.cc:1196] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/cf6c86b5e42340aa82299bce5701edcf
I20260809 20:22:14.248550 26849 ts_tablet_manager.cc:1939] T cf6c86b5e42340aa82299bce5701edcf P 3a663f4e52a2484b9e08bf94c8078ce5: Deleting consensus metadata
Aug 09 20:22:14 dist-test-slave-6k22 krb5kdc[26196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.1@KRBTEST.COM
I20260809 20:22:14.250273 26852 ts_tablet_manager.cc:1918] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:22:14.250258 26234 catalog_manager.cc:4841] TS 3a663f4e52a2484b9e08bf94c8078ce5 (127.5.61.1:37105): tablet cf6c86b5e42340aa82299bce5701edcf (table test-table [id=949658ba28b94fbf8f1fcc5b0a3a665c]) successfully deleted
I20260809 20:22:14.250532 26852 log.cc:1196] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/cf6c86b5e42340aa82299bce5701edcf
I20260809 20:22:14.251111 26852 ts_tablet_manager.cc:1939] T cf6c86b5e42340aa82299bce5701edcf P 9b760a97f421417dbd99a3a76c5b996e: Deleting consensus metadata
I20260809 20:22:14.252861 26231 catalog_manager.cc:4841] TS 9b760a97f421417dbd99a3a76c5b996e (127.5.61.3:39383): tablet cf6c86b5e42340aa82299bce5701edcf (table test-table [id=949658ba28b94fbf8f1fcc5b0a3a665c]) successfully deleted
I20260809 20:22:14.256059 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:14.224006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46896 (local address 127.5.61.1:37105)
0809 20:22:14.224184 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:14.224190 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:14.238277 (+ 14087us) server_negotiation.cc:407] Connection header received
0809 20:22:14.238532 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:14.238540 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:14.238653 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:14.238760 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:14.240717 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:14.241495 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:14.243453 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:14.244106 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:14.244218 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:14.249920 (+ 5702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:14.249931 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:14.249947 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:14.249988 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:14.253148 (+ 3160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:14.254015 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:14.254021 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:14.254023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:14.254140 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:14.254586 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:14.254590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:14.254591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:14.254920 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:14.255133 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:14.255516 (+ 383us) server_negotiation.cc:299] Negotiation successful
0809 20:22:14.255756 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 20:22:14.256790 26473 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:14.346917 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 26330
I20260809 20:22:14.369650 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 26487
Aug 09 20:22:14 dist-test-slave-6k22 krb5kdc[26196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.2@KRBTEST.COM
I20260809 20:22:14.385856 26817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:14.370479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41640 (local address 127.5.61.2:41697)
0809 20:22:14.370679 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:14.370685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:14.370806 (+ 121us) server_negotiation.cc:407] Connection header received
0809 20:22:14.371030 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:14.371036 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:14.371127 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:14.371228 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:14.372542 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:14.373220 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:14.374623 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:14.375097 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:14.375236 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:14.380347 (+ 5111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:14.380368 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:14.380384 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:14.380419 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:14.383099 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:14.383826 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:14.383834 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:14.383836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:14.383972 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:14.384370 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:14.384374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:14.384376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:14.384736 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:14.385040 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:14.385355 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 20:22:14.385582 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 20:22:14.386471 26635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:14.479461 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 26487
I20260809 20:22:14.508587 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 26647
Aug 09 20:22:14 dist-test-slave-6k22 krb5kdc[26196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.3@KRBTEST.COM
I20260809 20:22:14.526659 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:14.509453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60102 (local address 127.5.61.3:39383)
0809 20:22:14.509680 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:14.509685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:14.509877 (+ 192us) server_negotiation.cc:407] Connection header received
0809 20:22:14.510039 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:14.510045 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:14.510131 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:14.510259 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:14.511502 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:14.512748 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:14.514044 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:14.514766 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:14.514906 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:14.519530 (+ 4624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:14.519546 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:14.519601 (+ 55us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:14.519666 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:14.523725 (+ 4059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:14.524398 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:14.524411 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:14.524420 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:14.524586 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:14.524957 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:14.524965 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:14.524970 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:14.525474 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:14.525851 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:14.526014 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 20:22:14.526252 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 20:22:14.527478 26793 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:14.652124 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 26647
I20260809 20:22:14.676908 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 26210
I20260809 20:22:14.678020 26292 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:14.787369 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 26210
2026-08-09T20:22:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4796 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T20:22:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:22:14Z Disabled control of system clock
I20260809 20:22:14.877830 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:40107
--webserver_interface=127.5.61.62
--webserver_port=0
--builtin_ntp_servers=127.5.61.20:45187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:40107
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 20:22:15.207063 26862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:15.207314 26862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:15.207476 26862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:15.214617 26862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:22:15.214774 26862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:15.214865 26862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:15.214968 26862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:22:15.215054 26862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:22:15.223794 26862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:45187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:40107
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:40107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:15.224957 26862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:15.226761 26862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:15.238482 26872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:15.238605 26869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:15.238539 26870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:15.239681 26862 server_base.cc:1034] running on GCE node
I20260809 20:22:15.240379 26862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:15.241703 26862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:15.242959 26862 hybrid_clock.cc:648] HybridClock initialized: now 1786306935242905 us; error 44 us; skew 500 ppm
I20260809 20:22:15.243335 26862 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:22:15.250257 26862 webserver.cc:460] Webserver started at http://127.5.61.62:36909/ using document root <none> and password file <none>
I20260809 20:22:15.250960 26862 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:15.251096 26862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:15.251420 26862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:15.254356 26862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "028ff7c8647b46d78c85a57575c85ce3"
format_stamp: "Formatted at 2026-08-09 20:22:15 on dist-test-slave-6k22"
server_key: "7a287342a2e0095e83561ade67072394"
server_key_iv: "d38e42c3433465da9b7a2e81919db1d2"
server_key_version: "encryptionkey@0"
I20260809 20:22:15.255187 26862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "028ff7c8647b46d78c85a57575c85ce3"
format_stamp: "Formatted at 2026-08-09 20:22:15 on dist-test-slave-6k22"
server_key: "7a287342a2e0095e83561ade67072394"
server_key_iv: "d38e42c3433465da9b7a2e81919db1d2"
server_key_version: "encryptionkey@0"
I20260809 20:22:15.260170 26862 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260809 20:22:15.263525 26879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:15.264863 26862 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:22:15.265055 26862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "028ff7c8647b46d78c85a57575c85ce3"
format_stamp: "Formatted at 2026-08-09 20:22:15 on dist-test-slave-6k22"
server_key: "7a287342a2e0095e83561ade67072394"
server_key_iv: "d38e42c3433465da9b7a2e81919db1d2"
server_key_version: "encryptionkey@0"
I20260809 20:22:15.265256 26862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:15.285488 26862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:15.286273 26862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:15.286556 26862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:15.304879 26862 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:40107
I20260809 20:22:15.304898 26937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:40107 every 8 connection(s)
I20260809 20:22:15.306182 26862 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:22:15.309775 26938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:15.313670 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 26862
I20260809 20:22:15.313844 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:22:15.314303 5364 external_mini_cluster.cc:1428] Setting key 5002596888ca2374a97c30f44d2d09be
I20260809 20:22:15.322094 26938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 028ff7c8647b46d78c85a57575c85ce3: Bootstrap starting.
I20260809 20:22:15.325634 26938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 028ff7c8647b46d78c85a57575c85ce3: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:15.326960 26938 log.cc:824] T 00000000000000000000000000000000 P 028ff7c8647b46d78c85a57575c85ce3: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:15.330165 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:15.316159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50614 (local address 127.5.61.62:40107)
0809 20:22:15.317178 (+ 1019us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:15.317190 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:15.317244 (+ 54us) server_negotiation.cc:407] Connection header received
0809 20:22:15.320205 (+ 2961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:15.320233 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:15.320779 (+ 546us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:15.321232 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:15.321768 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:15.323750 (+ 1982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:15.325097 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:15.326186 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:15.326393 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:15.327578 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:15.327592 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:15.327615 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:15.328173 (+ 558us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:15.328297 (+ 124us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:15.328325 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:15.328435 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:15.328666 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 20:22:15.328967 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":309,"threads_started":1}
I20260809 20:22:15.330264 26938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 028ff7c8647b46d78c85a57575c85ce3: No bootstrap required, opened a new log
I20260809 20:22:15.337879 26938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 028ff7c8647b46d78c85a57575c85ce3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "028ff7c8647b46d78c85a57575c85ce3" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40107 } }
I20260809 20:22:15.338197 26938 raft_consensus.cc:374] T 00000000000000000000000000000000 P 028ff7c8647b46d78c85a57575c85ce3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:15.338258 26938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 028ff7c8647b46d78c85a57575c85ce3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 028ff7c8647b46d78c85a57575c85ce3, State: Initialized, Role: FOLLOWER
I20260809 20:22:15.338819 26938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 028ff7c8647b46d78c85a57575c85ce3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "028ff7c8647b46d78c85a57575c85ce3" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40107 } }
I20260809 20:22:15.339030 26938 raft_consensus.cc:388] T 00000000000000000000000000000000 P 028ff7c8647b46d78c85a57575c85ce3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:22:15.339126 26938 raft_consensus.cc:482] T 00000000000000000000000000000000 P 028ff7c8647b46d78c85a57575c85ce3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:22:15.339242 26938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 028ff7c8647b46d78c85a57575c85ce3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:15.341626 26938 raft_consensus.cc:504] T 00000000000000000000000000000000 P 028ff7c8647b46d78c85a57575c85ce3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "028ff7c8647b46d78c85a57575c85ce3" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40107 } }
I20260809 20:22:15.342101 26938 leader_election.cc:302] T 00000000000000000000000000000000 P 028ff7c8647b46d78c85a57575c85ce3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 028ff7c8647b46d78c85a57575c85ce3; no voters:
I20260809 20:22:15.342579 26938 leader_election.cc:288] T 00000000000000000000000000000000 P 028ff7c8647b46d78c85a57575c85ce3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:22:15.342732 26944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 028ff7c8647b46d78c85a57575c85ce3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:15.343463 26944 raft_consensus.cc:686] T 00000000000000000000000000000000 P 028ff7c8647b46d78c85a57575c85ce3 [term 1 LEADER]: Becoming Leader. State: Replica: 028ff7c8647b46d78c85a57575c85ce3, State: Running, Role: LEADER
I20260809 20:22:15.344030 26944 consensus_queue.cc:237] T 00000000000000000000000000000000 P 028ff7c8647b46d78c85a57575c85ce3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "028ff7c8647b46d78c85a57575c85ce3" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40107 } }
I20260809 20:22:15.344669 26938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 028ff7c8647b46d78c85a57575c85ce3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:22:15.347105 26946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 028ff7c8647b46d78c85a57575c85ce3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 028ff7c8647b46d78c85a57575c85ce3. Latest consensus state: current_term: 1 leader_uuid: "028ff7c8647b46d78c85a57575c85ce3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "028ff7c8647b46d78c85a57575c85ce3" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40107 } } }
I20260809 20:22:15.347851 26945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 028ff7c8647b46d78c85a57575c85ce3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "028ff7c8647b46d78c85a57575c85ce3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "028ff7c8647b46d78c85a57575c85ce3" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40107 } } }
I20260809 20:22:15.348197 26946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 028ff7c8647b46d78c85a57575c85ce3 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:15.348275 26945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 028ff7c8647b46d78c85a57575c85ce3 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:15.352478 26960 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:22:15.356997 26960 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:22:15.366318 26960 catalog_manager.cc:1347] Generated new cluster ID: 2bd746006a724b338b50e503987a90ae
I20260809 20:22:15.366438 26960 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:22:15.375305 26960 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:22:15.377010 26960 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:22:15.390431 26960 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 028ff7c8647b46d78c85a57575c85ce3: Generated new TSK 0
I20260809 20:22:15.391419 26960 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:22:15.404417 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:40107
--builtin_ntp_servers=127.5.61.20:45187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 20:22:15.743556 26974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:15.743847 26974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:15.744009 26974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:15.751399 26974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:15.751520 26974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:15.751695 26974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:22:15.760368 26974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:45187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:40107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:15.761546 26974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:15.763437 26974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:15.776449 26981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:15.776386 26982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:15.776364 26984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:15.777206 26974 server_base.cc:1034] running on GCE node
I20260809 20:22:15.777804 26974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:15.778716 26974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:15.779912 26974 hybrid_clock.cc:648] HybridClock initialized: now 1786306935779858 us; error 48 us; skew 500 ppm
I20260809 20:22:15.780224 26974 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:22:15.782436 26974 webserver.cc:460] Webserver started at http://127.5.61.1:46457/ using document root <none> and password file <none>
I20260809 20:22:15.783128 26974 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:15.783317 26974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:15.783782 26974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:15.786439 26974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "4d1ec78073dd44e9a5738c71b073dcbd"
format_stamp: "Formatted at 2026-08-09 20:22:15 on dist-test-slave-6k22"
server_key: "f617e400bf0ef2e572de2c7046c121fb"
server_key_iv: "62689d34d7ce83b314ad19a14bce136b"
server_key_version: "encryptionkey@0"
I20260809 20:22:15.787264 26974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "4d1ec78073dd44e9a5738c71b073dcbd"
format_stamp: "Formatted at 2026-08-09 20:22:15 on dist-test-slave-6k22"
server_key: "f617e400bf0ef2e572de2c7046c121fb"
server_key_iv: "62689d34d7ce83b314ad19a14bce136b"
server_key_version: "encryptionkey@0"
I20260809 20:22:15.792429 26974 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260809 20:22:15.795773 26990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:15.796922 26974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 20:22:15.797084 26974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "4d1ec78073dd44e9a5738c71b073dcbd"
format_stamp: "Formatted at 2026-08-09 20:22:15 on dist-test-slave-6k22"
server_key: "f617e400bf0ef2e572de2c7046c121fb"
server_key_iv: "62689d34d7ce83b314ad19a14bce136b"
server_key_version: "encryptionkey@0"
I20260809 20:22:15.797303 26974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:15.812041 26974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:15.812808 26974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:15.813061 26974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:15.813942 26974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:22:15.815300 26974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:22:15.815448 26974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:15.815601 26974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:22:15.815698 26974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260809 20:22:15.849169 26974 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:39763
I20260809 20:22:15.849231 27130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:39763 every 8 connection(s)
I20260809 20:22:15.850510 26974 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:22:15.854118 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 26974
I20260809 20:22:15.854328 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:22:15.854887 5364 external_mini_cluster.cc:1428] Setting key dc3dce2a9524d8cf58f4065a6ceb0bd1
I20260809 20:22:15.857654 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:40107
--builtin_ntp_servers=127.5.61.20:45187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 20:22:15.863324 27134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:15.853109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:54483 (local address 127.5.61.62:40107)
0809 20:22:15.853544 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:15.853549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:15.854896 (+ 1347us) server_negotiation.cc:407] Connection header received
0809 20:22:15.856968 (+ 2072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:15.856973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:15.857063 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:15.857185 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:15.858347 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:15.859038 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:15.860599 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:15.861021 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:15.861174 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:15.862484 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:15.862497 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:15.862500 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:15.862517 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:15.862557 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:15.862569 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:15.862661 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:15.862894 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 20:22:15.862982 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":244,"threads_started":1}
I20260809 20:22:15.864269 27136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:15.853616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40107 (local address 127.5.61.1:54483)
0809 20:22:15.854742 (+ 1126us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:15.854784 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:15.856726 (+ 1942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:15.857405 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:15.857413 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:15.857495 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:22:15.858185 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:15.858202 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:15.859181 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:15.859186 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:15.860457 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:15.860468 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:15.861244 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:15.861253 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:15.861491 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:15.862193 (+ 702us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 20:22:15.862196 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:22:15.862232 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 20:22:15.862301 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:22:15.862682 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:22:15.862699 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:22:15.862705 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 20:22:15.862884 (+ 179us) client_negotiation.cc:240] Negotiation successful
0809 20:22:15.863231 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":361,"threads_started":1}
I20260809 20:22:15.865756 27131 heartbeater.cc:347] Connected to a master server at 127.5.61.62:40107
I20260809 20:22:15.866024 27131 heartbeater.cc:464] Registering TS with master...
I20260809 20:22:15.866756 27131 heartbeater.cc:511] Master 127.5.61.62:40107 requested a full tablet report, sending...
I20260809 20:22:15.868743 26897 ts_manager.cc:194] Registered new tserver with Master: 4d1ec78073dd44e9a5738c71b073dcbd (127.5.61.1:39763)
I20260809 20:22:15.870373 26897 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.61.1:54483
W20260809 20:22:16.199359 27137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:16.199723 27137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:16.199824 27137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:16.206871 27137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:16.206961 27137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:16.207062 27137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:22:16.215705 27137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:45187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:40107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:16.216868 27137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:16.218535 27137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:16.231268 27142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:16.231429 27143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:16.231485 27145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:16.232295 27137 server_base.cc:1034] running on GCE node
I20260809 20:22:16.232784 27137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:16.233628 27137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:16.234820 27137 hybrid_clock.cc:648] HybridClock initialized: now 1786306936234772 us; error 53 us; skew 500 ppm
I20260809 20:22:16.235111 27137 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:22:16.237463 27137 webserver.cc:460] Webserver started at http://127.5.61.2:37301/ using document root <none> and password file <none>
I20260809 20:22:16.238081 27137 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:16.238168 27137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:16.238471 27137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:16.241268 27137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "87b83635a33c4c86854c8560da73b725"
format_stamp: "Formatted at 2026-08-09 20:22:16 on dist-test-slave-6k22"
server_key: "0aa891b8e6db8d23c0cf539929f444a2"
server_key_iv: "417d91cf6af2d64738026c45544cfd72"
server_key_version: "encryptionkey@0"
I20260809 20:22:16.242005 27137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "87b83635a33c4c86854c8560da73b725"
format_stamp: "Formatted at 2026-08-09 20:22:16 on dist-test-slave-6k22"
server_key: "0aa891b8e6db8d23c0cf539929f444a2"
server_key_iv: "417d91cf6af2d64738026c45544cfd72"
server_key_version: "encryptionkey@0"
I20260809 20:22:16.247185 27137 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 20:22:16.250801 27152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:16.252115 27137 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:22:16.252285 27137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "87b83635a33c4c86854c8560da73b725"
format_stamp: "Formatted at 2026-08-09 20:22:16 on dist-test-slave-6k22"
server_key: "0aa891b8e6db8d23c0cf539929f444a2"
server_key_iv: "417d91cf6af2d64738026c45544cfd72"
server_key_version: "encryptionkey@0"
I20260809 20:22:16.252516 27137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:16.281020 27137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:16.281880 27137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:16.282168 27137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:16.283165 27137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:22:16.284737 27137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:22:16.284873 27137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:16.284996 27137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:22:16.285079 27137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:16.324165 27137 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:34733
I20260809 20:22:16.324268 27293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:34733 every 8 connection(s)
I20260809 20:22:16.325433 27137 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:22:16.328354 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 27137
I20260809 20:22:16.328537 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:22:16.329106 5364 external_mini_cluster.cc:1428] Setting key 2082bb92ccf1a709eae579b303de6e88
I20260809 20:22:16.332078 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:40107
--builtin_ntp_servers=127.5.61.20:45187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 20:22:16.338313 27134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:16.328049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:44953 (local address 127.5.61.62:40107)
0809 20:22:16.328214 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:16.328218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:16.329984 (+ 1766us) server_negotiation.cc:407] Connection header received
0809 20:22:16.332024 (+ 2040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:16.332029 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:16.332118 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:16.332225 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:16.333260 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:16.333996 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:16.335512 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:16.335971 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:16.336116 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:16.337476 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:16.337480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:16.337483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:16.337501 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:16.337531 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:16.337542 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:16.337643 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:16.337914 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 20:22:16.338012 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 20:22:16.339274 27296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:16.328731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40107 (local address 127.5.61.2:44953)
0809 20:22:16.329795 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:16.329854 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:16.331751 (+ 1897us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:16.332470 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:16.332478 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:16.332561 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:22:16.333095 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:16.333110 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:16.334158 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:16.334169 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:16.335367 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:16.335379 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:16.336193 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:16.336200 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:16.336444 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:16.337145 (+ 701us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 20:22:16.337148 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:22:16.337189 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 20:22:16.337256 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:22:16.337644 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:22:16.337658 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:22:16.337667 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 20:22:16.337908 (+ 241us) client_negotiation.cc:240] Negotiation successful
0809 20:22:16.338280 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":313,"threads_started":1}
I20260809 20:22:16.340677 27294 heartbeater.cc:347] Connected to a master server at 127.5.61.62:40107
I20260809 20:22:16.340956 27294 heartbeater.cc:464] Registering TS with master...
I20260809 20:22:16.341606 27294 heartbeater.cc:511] Master 127.5.61.62:40107 requested a full tablet report, sending...
I20260809 20:22:16.342998 26897 ts_manager.cc:194] Registered new tserver with Master: 87b83635a33c4c86854c8560da73b725 (127.5.61.2:34733)
I20260809 20:22:16.343995 26897 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.61.2:44953
W20260809 20:22:16.671527 27297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:16.671828 27297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:16.671995 27297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:16.679049 27297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:16.679137 27297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:16.679306 27297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:22:16.687870 27297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:45187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:40107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:16.689069 27297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:16.690915 27297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:16.703913 27303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:16.704020 27306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:16.704031 27304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:16.704900 27297 server_base.cc:1034] running on GCE node
I20260809 20:22:16.705495 27297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:16.706425 27297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:16.707662 27297 hybrid_clock.cc:648] HybridClock initialized: now 1786306936707619 us; error 51 us; skew 500 ppm
I20260809 20:22:16.708014 27297 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:22:16.710402 27297 webserver.cc:460] Webserver started at http://127.5.61.3:45033/ using document root <none> and password file <none>
I20260809 20:22:16.711141 27297 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:16.711275 27297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:16.711653 27297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:16.714481 27297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "462dcefe31bf48a69eb206efda989117"
format_stamp: "Formatted at 2026-08-09 20:22:16 on dist-test-slave-6k22"
server_key: "199eb7af9933a918dc134e1faad237ea"
server_key_iv: "fd1605b68290fb155172246d1b57463a"
server_key_version: "encryptionkey@0"
I20260809 20:22:16.715273 27297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "462dcefe31bf48a69eb206efda989117"
format_stamp: "Formatted at 2026-08-09 20:22:16 on dist-test-slave-6k22"
server_key: "199eb7af9933a918dc134e1faad237ea"
server_key_iv: "fd1605b68290fb155172246d1b57463a"
server_key_version: "encryptionkey@0"
I20260809 20:22:16.720304 27297 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 20:22:16.723692 27314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:16.725059 27297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 20:22:16.725181 27297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "462dcefe31bf48a69eb206efda989117"
format_stamp: "Formatted at 2026-08-09 20:22:16 on dist-test-slave-6k22"
server_key: "199eb7af9933a918dc134e1faad237ea"
server_key_iv: "fd1605b68290fb155172246d1b57463a"
server_key_version: "encryptionkey@0"
I20260809 20:22:16.725358 27297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:16.797803 27297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:16.798650 27297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:16.798877 27297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:16.799880 27297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:22:16.801316 27297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:22:16.801419 27297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:16.801518 27297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:22:16.801571 27297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:16.834618 27297 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:44599
I20260809 20:22:16.834704 27447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:44599 every 8 connection(s)
I20260809 20:22:16.836043 27297 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:22:16.843375 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 27297
I20260809 20:22:16.843590 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:22:16.844058 5364 external_mini_cluster.cc:1428] Setting key 33b49d85b3198332f639643580f81dc0
I20260809 20:22:16.849853 27451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:16.838878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:56979 (local address 127.5.61.62:40107)
0809 20:22:16.839305 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:16.839310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:16.840505 (+ 1195us) server_negotiation.cc:407] Connection header received
0809 20:22:16.842252 (+ 1747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:16.842256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:16.842320 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:16.842415 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:16.843663 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:16.844614 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:16.846762 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:16.847258 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:16.847373 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:16.848862 (+ 1489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:16.848867 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:16.848872 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:16.848893 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:16.848944 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:16.848959 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:16.849055 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:16.849312 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 20:22:16.849406 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":203,"threads_started":1}
I20260809 20:22:16.850638 27452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:16.839403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:40107 (local address 127.5.61.3:56979)
0809 20:22:16.840342 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:16.840394 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:16.842015 (+ 1621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:16.842579 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:16.842586 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:16.842670 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:22:16.843470 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:16.843498 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:16.844788 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:16.844808 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:16.846587 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:16.846600 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:16.847488 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:16.847493 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:16.847765 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:16.848576 (+ 811us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 20:22:16.848578 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:22:16.848628 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 20:22:16.848699 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:22:16.849070 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:22:16.849080 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:22:16.849088 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 20:22:16.849239 (+ 151us) client_negotiation.cc:240] Negotiation successful
0809 20:22:16.849569 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":330,"threads_started":1}
I20260809 20:22:16.852031 27448 heartbeater.cc:347] Connected to a master server at 127.5.61.62:40107
I20260809 20:22:16.852320 27448 heartbeater.cc:464] Registering TS with master...
I20260809 20:22:16.853057 27448 heartbeater.cc:511] Master 127.5.61.62:40107 requested a full tablet report, sending...
I20260809 20:22:16.854413 26897 ts_manager.cc:194] Registered new tserver with Master: 462dcefe31bf48a69eb206efda989117 (127.5.61.3:44599)
I20260809 20:22:16.855172 26897 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.61.3:56979
I20260809 20:22:16.858917 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 20:22:16.868781 27451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:16.863077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50628 (local address 127.5.61.62:40107)
0809 20:22:16.863230 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:16.863234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:16.863726 (+ 492us) server_negotiation.cc:407] Connection header received
0809 20:22:16.863855 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:16.863859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:16.863925 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:16.864004 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:16.864444 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:16.865127 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:16.866470 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:16.866850 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:16.866936 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:16.868055 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:16.868060 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:16.868063 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:16.868078 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:16.868105 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:16.868116 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:16.868284 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:16.868399 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 20:22:16.868468 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 20:22:16.873188 27131 heartbeater.cc:503] Master 127.5.61.62:40107 was elected leader, sending a full tablet report...
I20260809 20:22:16.872440 26897 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50628:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:22:16.879093 26897 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:22:16.900249 27464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:16.890055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:34733 (local address 127.0.0.1:35022)
0809 20:22:16.891148 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:16.891197 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:16.891470 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:16.892566 (+ 1096us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:16.892575 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:16.892615 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:16.893128 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:16.893144 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:16.895261 (+ 2117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:16.895265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:16.897716 (+ 2451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:16.897738 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:16.899182 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:16.899188 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:16.899387 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:16.899445 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 20:22:16.899601 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 20:22:16.899819 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"mutex_wait_us":3,"thread_start_us":837,"threads_started":1}
I20260809 20:22:16.900301 27466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:16.890210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35022 (local address 127.5.61.2:34733)
0809 20:22:16.890826 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:16.890834 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:16.891315 (+ 481us) server_negotiation.cc:407] Connection header received
0809 20:22:16.891753 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:16.891761 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:16.892016 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:16.892189 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:16.893415 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:16.895086 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:16.897993 (+ 2907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:16.898980 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:16.899133 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:16.899387 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:16.899784 (+ 397us) server_negotiation.cc:299] Negotiation successful
0809 20:22:16.899935 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":325,"threads_started":1}
I20260809 20:22:16.902653 27467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:16.890709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:39763 (local address 127.0.0.1:57100)
0809 20:22:16.891435 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:16.891457 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:16.891626 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:16.894850 (+ 3224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:16.894855 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:16.894888 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:16.895261 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:16.895272 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:16.897218 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:16.897227 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:16.899754 (+ 2527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:16.899769 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:16.901811 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:16.901818 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:16.901998 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:16.902037 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 20:22:16.902129 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 20:22:16.902274 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"mutex_wait_us":76,"thread_start_us":2534,"threads_started":1}
I20260809 20:22:16.902711 27468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:16.891140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57100 (local address 127.5.61.1:39763)
0809 20:22:16.894284 (+ 3144us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:16.894296 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:16.894325 (+ 29us) server_negotiation.cc:407] Connection header received
0809 20:22:16.894419 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:16.894427 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:16.894647 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:16.894839 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:16.895486 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:16.897078 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:16.900137 (+ 3059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:16.901555 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:16.901747 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:16.902014 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:16.902170 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 20:22:16.902371 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2965,"thread_start_us":2839,"threads_started":1}
I20260809 20:22:16.903127 27469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:16.891711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:44599 (local address 127.0.0.1:54340)
0809 20:22:16.892290 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:16.892310 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:16.892434 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:16.895221 (+ 2787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:16.895225 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:16.895264 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:16.895732 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:16.895744 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:16.897503 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:16.897510 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:16.899754 (+ 2244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:16.899769 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:16.902107 (+ 2338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:16.902115 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:16.902303 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:16.902330 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 20:22:16.902412 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 20:22:16.902519 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":1,"thread_start_us":840,"threads_started":1}
I20260809 20:22:16.903702 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:16.892308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54340 (local address 127.5.61.3:44599)
0809 20:22:16.894683 (+ 2375us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:16.894692 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:16.894722 (+ 30us) server_negotiation.cc:407] Connection header received
0809 20:22:16.894832 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:16.894839 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:16.895031 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:16.895223 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:16.895916 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:16.897345 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:16.900667 (+ 3322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:16.901896 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:16.902750 (+ 854us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:16.903046 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:16.903118 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 20:22:16.903301 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2216,"thread_start_us":2042,"threads_started":1}
I20260809 20:22:16.905048 27215 tablet_service.cc:1467] Processing CreateTablet for tablet f7f27cb4a48d4f76aa4479d80258b10d (DEFAULT_TABLE table=test-table [id=0ec89545a5b94194bf41aa1ae0e6650d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:22:16.906615 27215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7f27cb4a48d4f76aa4479d80258b10d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:16.906484 27049 tablet_service.cc:1467] Processing CreateTablet for tablet f7f27cb4a48d4f76aa4479d80258b10d (DEFAULT_TABLE table=test-table [id=0ec89545a5b94194bf41aa1ae0e6650d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:22:16.907233 27366 tablet_service.cc:1467] Processing CreateTablet for tablet f7f27cb4a48d4f76aa4479d80258b10d (DEFAULT_TABLE table=test-table [id=0ec89545a5b94194bf41aa1ae0e6650d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:22:16.907859 27049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7f27cb4a48d4f76aa4479d80258b10d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:16.908407 27366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7f27cb4a48d4f76aa4479d80258b10d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:16.920286 27471 tablet_bootstrap.cc:492] T f7f27cb4a48d4f76aa4479d80258b10d P 462dcefe31bf48a69eb206efda989117: Bootstrap starting.
I20260809 20:22:16.922168 27472 tablet_bootstrap.cc:492] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725: Bootstrap starting.
I20260809 20:22:16.924392 27471 tablet_bootstrap.cc:654] T f7f27cb4a48d4f76aa4479d80258b10d P 462dcefe31bf48a69eb206efda989117: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:16.924728 27473 tablet_bootstrap.cc:492] T f7f27cb4a48d4f76aa4479d80258b10d P 4d1ec78073dd44e9a5738c71b073dcbd: Bootstrap starting.
I20260809 20:22:16.925681 27472 tablet_bootstrap.cc:654] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:16.925832 27471 log.cc:824] T f7f27cb4a48d4f76aa4479d80258b10d P 462dcefe31bf48a69eb206efda989117: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:16.926803 27472 log.cc:824] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:16.928023 27473 tablet_bootstrap.cc:654] T f7f27cb4a48d4f76aa4479d80258b10d P 4d1ec78073dd44e9a5738c71b073dcbd: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:16.929057 27471 tablet_bootstrap.cc:492] T f7f27cb4a48d4f76aa4479d80258b10d P 462dcefe31bf48a69eb206efda989117: No bootstrap required, opened a new log
I20260809 20:22:16.929087 27473 log.cc:824] T f7f27cb4a48d4f76aa4479d80258b10d P 4d1ec78073dd44e9a5738c71b073dcbd: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:16.929363 27471 ts_tablet_manager.cc:1397] T f7f27cb4a48d4f76aa4479d80258b10d P 462dcefe31bf48a69eb206efda989117: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 20:22:16.929977 27472 tablet_bootstrap.cc:492] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725: No bootstrap required, opened a new log
I20260809 20:22:16.930262 27472 ts_tablet_manager.cc:1397] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260809 20:22:16.931912 27473 tablet_bootstrap.cc:492] T f7f27cb4a48d4f76aa4479d80258b10d P 4d1ec78073dd44e9a5738c71b073dcbd: No bootstrap required, opened a new log
I20260809 20:22:16.932199 27473 ts_tablet_manager.cc:1397] T f7f27cb4a48d4f76aa4479d80258b10d P 4d1ec78073dd44e9a5738c71b073dcbd: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260809 20:22:16.938761 27473 raft_consensus.cc:348] T f7f27cb4a48d4f76aa4479d80258b10d P 4d1ec78073dd44e9a5738c71b073dcbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b83635a33c4c86854c8560da73b725" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34733 } } peers { permanent_uuid: "462dcefe31bf48a69eb206efda989117" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 44599 } } peers { permanent_uuid: "4d1ec78073dd44e9a5738c71b073dcbd" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 39763 } }
I20260809 20:22:16.939114 27473 raft_consensus.cc:374] T f7f27cb4a48d4f76aa4479d80258b10d P 4d1ec78073dd44e9a5738c71b073dcbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:16.939248 27473 raft_consensus.cc:729] T f7f27cb4a48d4f76aa4479d80258b10d P 4d1ec78073dd44e9a5738c71b073dcbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d1ec78073dd44e9a5738c71b073dcbd, State: Initialized, Role: FOLLOWER
I20260809 20:22:16.939877 27473 consensus_queue.cc:260] T f7f27cb4a48d4f76aa4479d80258b10d P 4d1ec78073dd44e9a5738c71b073dcbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b83635a33c4c86854c8560da73b725" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34733 } } peers { permanent_uuid: "462dcefe31bf48a69eb206efda989117" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 44599 } } peers { permanent_uuid: "4d1ec78073dd44e9a5738c71b073dcbd" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 39763 } }
I20260809 20:22:16.940030 27471 raft_consensus.cc:348] T f7f27cb4a48d4f76aa4479d80258b10d P 462dcefe31bf48a69eb206efda989117 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b83635a33c4c86854c8560da73b725" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34733 } } peers { permanent_uuid: "462dcefe31bf48a69eb206efda989117" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 44599 } } peers { permanent_uuid: "4d1ec78073dd44e9a5738c71b073dcbd" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 39763 } }
I20260809 20:22:16.940457 27471 raft_consensus.cc:374] T f7f27cb4a48d4f76aa4479d80258b10d P 462dcefe31bf48a69eb206efda989117 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:16.940565 27471 raft_consensus.cc:729] T f7f27cb4a48d4f76aa4479d80258b10d P 462dcefe31bf48a69eb206efda989117 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 462dcefe31bf48a69eb206efda989117, State: Initialized, Role: FOLLOWER
I20260809 20:22:16.940678 27472 raft_consensus.cc:348] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b83635a33c4c86854c8560da73b725" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34733 } } peers { permanent_uuid: "462dcefe31bf48a69eb206efda989117" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 44599 } } peers { permanent_uuid: "4d1ec78073dd44e9a5738c71b073dcbd" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 39763 } }
I20260809 20:22:16.941102 27472 raft_consensus.cc:374] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:16.941203 27472 raft_consensus.cc:729] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87b83635a33c4c86854c8560da73b725, State: Initialized, Role: FOLLOWER
I20260809 20:22:16.941792 27473 ts_tablet_manager.cc:1428] T f7f27cb4a48d4f76aa4479d80258b10d P 4d1ec78073dd44e9a5738c71b073dcbd: Time spent starting tablet: real 0.009s user 0.006s sys 0.003s
I20260809 20:22:16.941792 27472 consensus_queue.cc:260] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b83635a33c4c86854c8560da73b725" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34733 } } peers { permanent_uuid: "462dcefe31bf48a69eb206efda989117" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 44599 } } peers { permanent_uuid: "4d1ec78073dd44e9a5738c71b073dcbd" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 39763 } }
I20260809 20:22:16.942833 27471 consensus_queue.cc:260] T f7f27cb4a48d4f76aa4479d80258b10d P 462dcefe31bf48a69eb206efda989117 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b83635a33c4c86854c8560da73b725" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34733 } } peers { permanent_uuid: "462dcefe31bf48a69eb206efda989117" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 44599 } } peers { permanent_uuid: "4d1ec78073dd44e9a5738c71b073dcbd" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 39763 } }
I20260809 20:22:16.943560 27472 ts_tablet_manager.cc:1428] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260809 20:22:16.943964 27448 heartbeater.cc:503] Master 127.5.61.62:40107 was elected leader, sending a full tablet report...
I20260809 20:22:16.944065 27294 heartbeater.cc:503] Master 127.5.61.62:40107 was elected leader, sending a full tablet report...
I20260809 20:22:16.945417 27471 ts_tablet_manager.cc:1428] T f7f27cb4a48d4f76aa4479d80258b10d P 462dcefe31bf48a69eb206efda989117: Time spent starting tablet: real 0.016s user 0.013s sys 0.000s
W20260809 20:22:17.077649 27295 tablet.cc:2369] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:22:17.078593 27479 raft_consensus.cc:482] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:22:17.078868 27479 raft_consensus.cc:504] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b83635a33c4c86854c8560da73b725" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34733 } } peers { permanent_uuid: "462dcefe31bf48a69eb206efda989117" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 44599 } } peers { permanent_uuid: "4d1ec78073dd44e9a5738c71b073dcbd" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 39763 } }
I20260809 20:22:17.080570 27479 leader_election.cc:288] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 462dcefe31bf48a69eb206efda989117 (127.5.61.3:44599), 4d1ec78073dd44e9a5738c71b073dcbd (127.5.61.1:39763)
W20260809 20:22:17.087962 27450 tablet.cc:2369] T f7f27cb4a48d4f76aa4479d80258b10d P 462dcefe31bf48a69eb206efda989117: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:22:17.088289 27481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:17.081010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:44599 (local address 127.5.61.2:47039)
0809 20:22:17.082158 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:17.082182 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:17.082353 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:17.082781 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:17.082785 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:17.082809 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:17.083297 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:17.083310 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:17.084530 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:17.084537 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:17.086139 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:17.086151 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:17.087368 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:17.087376 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:17.087644 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:17.087682 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 20:22:17.087788 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 20:22:17.087910 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1003,"mutex_wait_us":134,"thread_start_us":557,"threads_started":1}
I20260809 20:22:17.088395 27468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:17.081067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:46983 (local address 127.5.61.1:39763)
0809 20:22:17.081308 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:17.081317 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:17.082342 (+ 1025us) server_negotiation.cc:407] Connection header received
0809 20:22:17.082674 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:17.082681 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:17.082790 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:17.082880 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:17.083645 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:17.084505 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:17.086323 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:17.087133 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:17.087274 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:17.087384 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:17.088035 (+ 651us) server_negotiation.cc:299] Negotiation successful
0809 20:22:17.088157 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 20:22:17.088745 27482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:17.081142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:39763 (local address 127.5.61.2:46983)
0809 20:22:17.082205 (+ 1063us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:17.082220 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:17.082489 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:17.082920 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:17.082924 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:17.082962 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:17.083419 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:17.083431 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:17.084646 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:17.084650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:17.086137 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:17.086149 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:17.087330 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:17.087336 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:17.087548 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:17.087657 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 20:22:17.087791 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 20:22:17.087916 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":947,"mutex_wait_us":60,"thread_start_us":637,"threads_started":1}
I20260809 20:22:17.089078 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:17.081090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:47039 (local address 127.5.61.3:44599)
0809 20:22:17.081303 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:17.081312 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:17.082304 (+ 992us) server_negotiation.cc:407] Connection header received
0809 20:22:17.082538 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:17.082547 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:17.082631 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:17.082738 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:17.083469 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:17.084391 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:17.086401 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:17.087137 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:17.087285 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:17.087384 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:17.087819 (+ 435us) server_negotiation.cc:299] Negotiation successful
0809 20:22:17.087943 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 20:22:17.089730 27073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7f27cb4a48d4f76aa4479d80258b10d" candidate_uuid: "87b83635a33c4c86854c8560da73b725" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d1ec78073dd44e9a5738c71b073dcbd" is_pre_election: true
I20260809 20:22:17.089876 27395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7f27cb4a48d4f76aa4479d80258b10d" candidate_uuid: "87b83635a33c4c86854c8560da73b725" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "462dcefe31bf48a69eb206efda989117" is_pre_election: true
I20260809 20:22:17.090257 27073 raft_consensus.cc:2454] T f7f27cb4a48d4f76aa4479d80258b10d P 4d1ec78073dd44e9a5738c71b073dcbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87b83635a33c4c86854c8560da73b725 in term 0.
I20260809 20:22:17.090332 27395 raft_consensus.cc:2454] T f7f27cb4a48d4f76aa4479d80258b10d P 462dcefe31bf48a69eb206efda989117 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87b83635a33c4c86854c8560da73b725 in term 0.
I20260809 20:22:17.090938 27156 leader_election.cc:302] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d1ec78073dd44e9a5738c71b073dcbd, 87b83635a33c4c86854c8560da73b725; no voters:
I20260809 20:22:17.091298 27479 raft_consensus.cc:2781] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:22:17.091429 27479 raft_consensus.cc:482] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:22:17.091548 27479 raft_consensus.cc:3037] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:17.093622 27479 raft_consensus.cc:504] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b83635a33c4c86854c8560da73b725" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34733 } } peers { permanent_uuid: "462dcefe31bf48a69eb206efda989117" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 44599 } } peers { permanent_uuid: "4d1ec78073dd44e9a5738c71b073dcbd" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 39763 } }
I20260809 20:22:17.094244 27479 leader_election.cc:288] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725 [CANDIDATE]: Term 1 election: Requested vote from peers 462dcefe31bf48a69eb206efda989117 (127.5.61.3:44599), 4d1ec78073dd44e9a5738c71b073dcbd (127.5.61.1:39763)
I20260809 20:22:17.094570 27395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7f27cb4a48d4f76aa4479d80258b10d" candidate_uuid: "87b83635a33c4c86854c8560da73b725" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "462dcefe31bf48a69eb206efda989117"
I20260809 20:22:17.094777 27395 raft_consensus.cc:3037] T f7f27cb4a48d4f76aa4479d80258b10d P 462dcefe31bf48a69eb206efda989117 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:17.094791 27073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7f27cb4a48d4f76aa4479d80258b10d" candidate_uuid: "87b83635a33c4c86854c8560da73b725" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d1ec78073dd44e9a5738c71b073dcbd"
I20260809 20:22:17.095006 27073 raft_consensus.cc:3037] T f7f27cb4a48d4f76aa4479d80258b10d P 4d1ec78073dd44e9a5738c71b073dcbd [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:17.096972 27395 raft_consensus.cc:2454] T f7f27cb4a48d4f76aa4479d80258b10d P 462dcefe31bf48a69eb206efda989117 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87b83635a33c4c86854c8560da73b725 in term 1.
I20260809 20:22:17.097265 27073 raft_consensus.cc:2454] T f7f27cb4a48d4f76aa4479d80258b10d P 4d1ec78073dd44e9a5738c71b073dcbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87b83635a33c4c86854c8560da73b725 in term 1.
I20260809 20:22:17.097400 27154 leader_election.cc:302] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 462dcefe31bf48a69eb206efda989117, 87b83635a33c4c86854c8560da73b725; no voters:
I20260809 20:22:17.097690 27479 raft_consensus.cc:2781] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:17.098143 27479 raft_consensus.cc:686] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725 [term 1 LEADER]: Becoming Leader. State: Replica: 87b83635a33c4c86854c8560da73b725, State: Running, Role: LEADER
I20260809 20:22:17.098703 27479 consensus_queue.cc:237] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b83635a33c4c86854c8560da73b725" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34733 } } peers { permanent_uuid: "462dcefe31bf48a69eb206efda989117" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 44599 } } peers { permanent_uuid: "4d1ec78073dd44e9a5738c71b073dcbd" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 39763 } }
W20260809 20:22:17.102782 27132 tablet.cc:2369] T f7f27cb4a48d4f76aa4479d80258b10d P 4d1ec78073dd44e9a5738c71b073dcbd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:22:17.103093 26895 catalog_manager.cc:5495] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725 reported cstate change: term changed from 0 to 1, leader changed from <none> to 87b83635a33c4c86854c8560da73b725 (127.5.61.2). New cstate: current_term: 1 leader_uuid: "87b83635a33c4c86854c8560da73b725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87b83635a33c4c86854c8560da73b725" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 34733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "462dcefe31bf48a69eb206efda989117" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 44599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d1ec78073dd44e9a5738c71b073dcbd" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 39763 } health_report { overall_health: UNKNOWN } } }
I20260809 20:22:17.141798 27466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:17.136059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35030 (local address 127.5.61.2:34733)
0809 20:22:17.136270 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:17.136276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:17.136399 (+ 123us) server_negotiation.cc:407] Connection header received
0809 20:22:17.136638 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:17.136642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:17.136739 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:17.136901 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:22:17.137280 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:17.137964 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:17.139464 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:17.139961 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:17.140108 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:17.140700 (+ 592us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:22:17.141173 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:22:17.141382 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:17.141509 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 20:22:17.141551 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 20:22:17.149987 27073 raft_consensus.cc:1261] T f7f27cb4a48d4f76aa4479d80258b10d P 4d1ec78073dd44e9a5738c71b073dcbd [term 1 FOLLOWER]: Refusing update from remote peer 87b83635a33c4c86854c8560da73b725: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:22:17.149987 27395 raft_consensus.cc:1261] T f7f27cb4a48d4f76aa4479d80258b10d P 462dcefe31bf48a69eb206efda989117 [term 1 FOLLOWER]: Refusing update from remote peer 87b83635a33c4c86854c8560da73b725: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:22:17.150949 27483 consensus_queue.cc:1035] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725 [LEADER]: Connected to new peer: Peer: permanent_uuid: "462dcefe31bf48a69eb206efda989117" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 44599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:22:17.151266 27479 consensus_queue.cc:1035] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d1ec78073dd44e9a5738c71b073dcbd" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 39763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:22:17.168866 27487 mvcc.cc:205] Tried to move back new op lower bound from 7316713214556508160 to 7316713214360350720. Current Snapshot: MvccSnapshot[applied={T|T < 7316713214556508160}]
I20260809 20:22:17.177829 26896 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:22:17.178115 26896 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:22:17.181946 26896 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 028ff7c8647b46d78c85a57575c85ce3: Sending DeleteTablet for 3 replicas of tablet f7f27cb4a48d4f76aa4479d80258b10d
I20260809 20:22:17.183115 27049 tablet_service.cc:1514] Processing DeleteTablet for tablet f7f27cb4a48d4f76aa4479d80258b10d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:22:17 UTC) from {username='slave'} at 127.0.0.1:57100
I20260809 20:22:17.183316 27215 tablet_service.cc:1514] Processing DeleteTablet for tablet f7f27cb4a48d4f76aa4479d80258b10d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:22:17 UTC) from {username='slave'} at 127.0.0.1:35022
I20260809 20:22:17.184005 27366 tablet_service.cc:1514] Processing DeleteTablet for tablet f7f27cb4a48d4f76aa4479d80258b10d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:22:17 UTC) from {username='slave'} at 127.0.0.1:54340
I20260809 20:22:17.184332 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 26974
I20260809 20:22:17.198338 27497 tablet_replica.cc:330] T f7f27cb4a48d4f76aa4479d80258b10d P 4d1ec78073dd44e9a5738c71b073dcbd: stopping tablet replica
I20260809 20:22:17.198837 27497 raft_consensus.cc:2229] T f7f27cb4a48d4f76aa4479d80258b10d P 4d1ec78073dd44e9a5738c71b073dcbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:22:17.199410 27497 raft_consensus.cc:2258] T f7f27cb4a48d4f76aa4479d80258b10d P 4d1ec78073dd44e9a5738c71b073dcbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:22:17.200323 27499 tablet_replica.cc:330] T f7f27cb4a48d4f76aa4479d80258b10d P 462dcefe31bf48a69eb206efda989117: stopping tablet replica
I20260809 20:22:17.200688 27499 raft_consensus.cc:2229] T f7f27cb4a48d4f76aa4479d80258b10d P 462dcefe31bf48a69eb206efda989117 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:22:17.201233 27499 raft_consensus.cc:2258] T f7f27cb4a48d4f76aa4479d80258b10d P 462dcefe31bf48a69eb206efda989117 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:22:17.203015 27498 tablet_replica.cc:330] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725: stopping tablet replica
I20260809 20:22:17.203543 27499 ts_tablet_manager.cc:1905] T f7f27cb4a48d4f76aa4479d80258b10d P 462dcefe31bf48a69eb206efda989117: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:22:17.203511 27498 raft_consensus.cc:2229] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725 [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:22:17.204330 27498 raft_consensus.cc:2258] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:22:17.205771 27468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:17.186436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57116 (local address 127.5.61.1:39763)
0809 20:22:17.186652 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:17.186660 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:17.197082 (+ 10422us) server_negotiation.cc:407] Connection header received
0809 20:22:17.197603 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:17.197611 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:17.197770 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:17.197874 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:17.198605 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:17.199845 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:17.201857 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:17.202521 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:17.202627 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:17.204148 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:17.204161 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:17.204179 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:17.204890 (+ 711us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:17.204943 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:17.204967 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:17.205052 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:17.205317 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 20:22:17.205440 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 20:22:17.206772 27498 ts_tablet_manager.cc:1905] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:22:17.206851 27126 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:17.212292 27499 ts_tablet_manager.cc:1918] T f7f27cb4a48d4f76aa4479d80258b10d P 462dcefe31bf48a69eb206efda989117: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:22:17.212489 27499 log.cc:1196] T f7f27cb4a48d4f76aa4479d80258b10d P 462dcefe31bf48a69eb206efda989117: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/f7f27cb4a48d4f76aa4479d80258b10d
I20260809 20:22:17.213132 27499 ts_tablet_manager.cc:1939] T f7f27cb4a48d4f76aa4479d80258b10d P 462dcefe31bf48a69eb206efda989117: Deleting consensus metadata
I20260809 20:22:17.214505 27498 ts_tablet_manager.cc:1918] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:22:17.214751 27498 log.cc:1196] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/f7f27cb4a48d4f76aa4479d80258b10d
I20260809 20:22:17.215370 27498 ts_tablet_manager.cc:1939] T f7f27cb4a48d4f76aa4479d80258b10d P 87b83635a33c4c86854c8560da73b725: Deleting consensus metadata
I20260809 20:22:17.215356 26882 catalog_manager.cc:4841] TS 462dcefe31bf48a69eb206efda989117 (127.5.61.3:44599): tablet f7f27cb4a48d4f76aa4479d80258b10d (table test-table [id=0ec89545a5b94194bf41aa1ae0e6650d]) successfully deleted
I20260809 20:22:17.217176 26882 catalog_manager.cc:4841] TS 87b83635a33c4c86854c8560da73b725 (127.5.61.2:34733): tablet f7f27cb4a48d4f76aa4479d80258b10d (table test-table [id=0ec89545a5b94194bf41aa1ae0e6650d]) successfully deleted
I20260809 20:22:17.345139 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 26974
I20260809 20:22:17.374753 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 27137
W20260809 20:22:17.374768 26884 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.61.1:39763 (error 108)
I20260809 20:22:17.376818 27469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:17.376264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:39763 (local address 127.0.0.1:57130)
0809 20:22:17.376455 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:17.376614 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.1:39763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 20:22:17.377096 26884 catalog_manager.cc:4584] TS 4d1ec78073dd44e9a5738c71b073dcbd (127.5.61.1:39763): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f7f27cb4a48d4f76aa4479d80258b10d: Network error: Client connection negotiation failed: client connection to 127.5.61.1:39763: connect: Connection refused (error 111)
I20260809 20:22:17.382357 27466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:17.375960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35044 (local address 127.5.61.2:34733)
0809 20:22:17.376151 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:17.376157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:17.376177 (+ 20us) server_negotiation.cc:407] Connection header received
0809 20:22:17.376256 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:17.376261 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:17.376374 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:17.376516 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:17.376958 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:17.377823 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:17.379186 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:17.379688 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:17.379839 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:17.380975 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:17.380983 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:17.380996 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:17.381601 (+ 605us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:17.381645 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:17.381663 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:17.381800 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:17.381934 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 20:22:17.382035 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 20:22:17.383067 27288 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:17.430660 27469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:17.430119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:39763 (local address 127.0.0.1:57142)
0809 20:22:17.430354 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:17.430472 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.1:39763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 20:22:17.431744 27469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:17.431329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:39763 (local address 127.0.0.1:57158)
0809 20:22:17.431484 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:17.431593 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.1:39763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 20:22:17.492427 27469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:17.491801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:39763 (local address 127.0.0.1:57164)
0809 20:22:17.492070 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:17.492186 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.1:39763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260809 20:22:17.493505 27469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:17.493113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:39763 (local address 127.0.0.1:57174)
0809 20:22:17.493275 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:17.493363 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.1:39763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 20:22:17.497627 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 27137
I20260809 20:22:17.520932 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 27297
I20260809 20:22:17.527885 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:17.521680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54346 (local address 127.5.61.3:44599)
0809 20:22:17.521878 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:17.521886 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:17.521958 (+ 72us) server_negotiation.cc:407] Connection header received
0809 20:22:17.522170 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:17.522175 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:17.522259 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:17.522365 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:17.522784 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:17.523497 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:17.524879 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:17.525253 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:17.525385 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:17.526494 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:17.526503 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:17.526518 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:17.527063 (+ 545us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:17.527114 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:17.527133 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:17.527257 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:17.527424 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 20:22:17.527531 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 20:22:17.528486 27442 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:17.574112 27469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:17.573597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:39763 (local address 127.0.0.1:57178)
0809 20:22:17.573831 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:17.573934 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.1:39763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260809 20:22:17.575253 27469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:17.574846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:39763 (local address 127.0.0.1:57188)
0809 20:22:17.575071 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:17.575148 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.1:39763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260809 20:22:17.616655 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 27297
I20260809 20:22:17.640543 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 26862
I20260809 20:22:17.641594 26933 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:17.749082 27469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:17.748405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:39763 (local address 127.0.0.1:57194)
0809 20:22:17.748728 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:17.748858 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.1:39763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
I20260809 20:22:17.749154 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 26862
2026-08-09T20:22:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2956 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:22:17 dist-test-slave-6k22 krb5kdc[27509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:22:17 dist-test-slave-6k22 krb5kdc[27509](info): set up 2 sockets
Aug 09 20:22:17 dist-test-slave-6k22 krb5kdc[27509](info): commencing operation
krb5kdc: starting...
W20260809 20:22:18.939389 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:22:19 dist-test-slave-6k22 krb5kdc[27509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:22:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:22:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:22:19.223155 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:36215
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:41233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:36215
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:19.566334 27526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:19.566607 27526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:19.566787 27526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:19.573957 27526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:22:19.574088 27526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:19.574175 27526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:19.574283 27526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:22:19.574368 27526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:22:19.584164 27526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:41233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:36215
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:36215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:19.585527 27526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:19.586892 27526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:19.599285 27532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:19.599197 27533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:19.599131 27535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:19.600452 27526 server_base.cc:1034] running on GCE node
I20260809 20:22:19.601192 27526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:19.602577 27526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:19.603811 27526 hybrid_clock.cc:648] HybridClock initialized: now 1786306939603766 us; error 49 us; skew 500 ppm
I20260809 20:22:19.604182 27526 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:19 dist-test-slave-6k22 krb5kdc[27509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306939, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:19.607173 27526 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:22:19.608695 27526 webserver.cc:460] Webserver started at http://127.5.61.62:37259/ using document root <none> and password file <none>
I20260809 20:22:19.609369 27526 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:19.609510 27526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:19.609819 27526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:19.612805 27526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "42ef4160d4cf47379fe6b4faa9b7cbf9"
format_stamp: "Formatted at 2026-08-09 20:22:19 on dist-test-slave-6k22"
server_key: "1eccf279c9fc9dbfb4f70af697d16168"
server_key_iv: "f165af0bb37e33f1bb68a8ba7b7aac23"
server_key_version: "encryptionkey@0"
I20260809 20:22:19.613559 27526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "42ef4160d4cf47379fe6b4faa9b7cbf9"
format_stamp: "Formatted at 2026-08-09 20:22:19 on dist-test-slave-6k22"
server_key: "1eccf279c9fc9dbfb4f70af697d16168"
server_key_iv: "f165af0bb37e33f1bb68a8ba7b7aac23"
server_key_version: "encryptionkey@0"
I20260809 20:22:19.618718 27526 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 20:22:19.622095 27545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:19.623395 27526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 20:22:19.623641 27526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "42ef4160d4cf47379fe6b4faa9b7cbf9"
format_stamp: "Formatted at 2026-08-09 20:22:19 on dist-test-slave-6k22"
server_key: "1eccf279c9fc9dbfb4f70af697d16168"
server_key_iv: "f165af0bb37e33f1bb68a8ba7b7aac23"
server_key_version: "encryptionkey@0"
I20260809 20:22:19.623852 27526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:19.656873 27526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:19.660964 27526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:19.661273 27526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:19.680291 27526 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:36215
I20260809 20:22:19.680334 27607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:36215 every 8 connection(s)
I20260809 20:22:19.681631 27526 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:22:19.685232 27608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:19.686056 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 27526
I20260809 20:22:19.686251 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:22:19.686703 5364 external_mini_cluster.cc:1428] Setting key 34e6d853e3d6b7959edd20dcbdfb4b42
I20260809 20:22:19.699793 27608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42ef4160d4cf47379fe6b4faa9b7cbf9: Bootstrap starting.
Aug 09 20:22:19 dist-test-slave-6k22 krb5kdc[27509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:22:19.702979 27608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42ef4160d4cf47379fe6b4faa9b7cbf9: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:19.704464 27608 log.cc:824] T 00000000000000000000000000000000 P 42ef4160d4cf47379fe6b4faa9b7cbf9: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:19.707924 27608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42ef4160d4cf47379fe6b4faa9b7cbf9: No bootstrap required, opened a new log
I20260809 20:22:19.711802 27611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:19.688708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54964 (local address 127.5.61.62:36215)
0809 20:22:19.689508 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:19.689522 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:19.689578 (+ 56us) server_negotiation.cc:407] Connection header received
0809 20:22:19.691957 (+ 2379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:19.691985 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:19.692461 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:19.692912 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:19.694170 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:19.696250 (+ 2080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:19.697515 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:19.698546 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:19.698721 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:19.702769 (+ 4048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:19.702782 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:19.702802 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:19.702853 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:19.707097 (+ 4244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:19.707879 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:19.707902 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:19.707917 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:19.708150 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:19.708538 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:19.708543 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:19.708547 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:19.709295 (+ 748us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:19.709588 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:19.709996 (+ 408us) server_negotiation.cc:299] Negotiation successful
0809 20:22:19.710460 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":349,"threads_started":1}
I20260809 20:22:19.717312 27608 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42ef4160d4cf47379fe6b4faa9b7cbf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42ef4160d4cf47379fe6b4faa9b7cbf9" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 36215 } }
I20260809 20:22:19.717759 27608 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42ef4160d4cf47379fe6b4faa9b7cbf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:19.717885 27608 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42ef4160d4cf47379fe6b4faa9b7cbf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42ef4160d4cf47379fe6b4faa9b7cbf9, State: Initialized, Role: FOLLOWER
I20260809 20:22:19.718493 27608 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42ef4160d4cf47379fe6b4faa9b7cbf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42ef4160d4cf47379fe6b4faa9b7cbf9" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 36215 } }
I20260809 20:22:19.718771 27608 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42ef4160d4cf47379fe6b4faa9b7cbf9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:22:19.718919 27608 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42ef4160d4cf47379fe6b4faa9b7cbf9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:22:19.719101 27608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42ef4160d4cf47379fe6b4faa9b7cbf9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:19.721553 27608 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42ef4160d4cf47379fe6b4faa9b7cbf9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42ef4160d4cf47379fe6b4faa9b7cbf9" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 36215 } }
I20260809 20:22:19.722047 27608 leader_election.cc:302] T 00000000000000000000000000000000 P 42ef4160d4cf47379fe6b4faa9b7cbf9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42ef4160d4cf47379fe6b4faa9b7cbf9; no voters:
I20260809 20:22:19.722607 27608 leader_election.cc:288] T 00000000000000000000000000000000 P 42ef4160d4cf47379fe6b4faa9b7cbf9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:22:19.722764 27613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42ef4160d4cf47379fe6b4faa9b7cbf9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:19.723453 27613 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42ef4160d4cf47379fe6b4faa9b7cbf9 [term 1 LEADER]: Becoming Leader. State: Replica: 42ef4160d4cf47379fe6b4faa9b7cbf9, State: Running, Role: LEADER
I20260809 20:22:19.724046 27613 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42ef4160d4cf47379fe6b4faa9b7cbf9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42ef4160d4cf47379fe6b4faa9b7cbf9" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 36215 } }
I20260809 20:22:19.724859 27608 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42ef4160d4cf47379fe6b4faa9b7cbf9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:22:19.727476 27614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42ef4160d4cf47379fe6b4faa9b7cbf9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42ef4160d4cf47379fe6b4faa9b7cbf9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42ef4160d4cf47379fe6b4faa9b7cbf9" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 36215 } } }
I20260809 20:22:19.727470 27616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42ef4160d4cf47379fe6b4faa9b7cbf9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42ef4160d4cf47379fe6b4faa9b7cbf9. Latest consensus state: current_term: 1 leader_uuid: "42ef4160d4cf47379fe6b4faa9b7cbf9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42ef4160d4cf47379fe6b4faa9b7cbf9" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 36215 } } }
I20260809 20:22:19.727978 27614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42ef4160d4cf47379fe6b4faa9b7cbf9 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:19.727993 27616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42ef4160d4cf47379fe6b4faa9b7cbf9 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:19.729506 27622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:22:19.734629 27622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:22:19.743182 27622 catalog_manager.cc:1347] Generated new cluster ID: fa2c1faaf5074dd1918345b64f30661c
I20260809 20:22:19.743309 27622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:22:19.763808 27622 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:22:19.765094 27622 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:22:19.772974 27622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42ef4160d4cf47379fe6b4faa9b7cbf9: Generated new TSK 0
I20260809 20:22:19.773967 27622 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.61.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
Entry for principal kudu/127.5.61.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab.
I20260809 20:22:19.859803 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:36215
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:41233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:20.204641 27639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:20.204918 27639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:20.205093 27639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:20.212182 27639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:20.212286 27639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:20.212397 27639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:22:20.221761 27639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:41233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.1.keytab
--principal=kudu/127.5.61.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:36215
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:20.222839 27639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:20.224186 27639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:20.238214 27645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:20.238183 27646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:20.238261 27649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:20.239475 27639 server_base.cc:1034] running on GCE node
I20260809 20:22:20.240091 27639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:20.240896 27639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:20.242093 27639 hybrid_clock.cc:648] HybridClock initialized: now 1786306940242048 us; error 41 us; skew 500 ppm
I20260809 20:22:20.242439 27639 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:20 dist-test-slave-6k22 krb5kdc[27509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306940, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:20.245575 27639 init.cc:377] Logged in from keytab as kudu/127.5.61.1@KRBTEST.COM (short username kudu)
I20260809 20:22:20.247028 27639 webserver.cc:460] Webserver started at http://127.5.61.1:36947/ using document root <none> and password file <none>
I20260809 20:22:20.247704 27639 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:20.247813 27639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:20.248081 27639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:20.250780 27639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "1dad334386314e3ea7a8736a9d312136"
format_stamp: "Formatted at 2026-08-09 20:22:20 on dist-test-slave-6k22"
server_key: "cb5bfeaf63098f9283427d6946b77d20"
server_key_iv: "43c14c62a33f477a59a12aa1b29ea4e3"
server_key_version: "encryptionkey@0"
I20260809 20:22:20.251473 27639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "1dad334386314e3ea7a8736a9d312136"
format_stamp: "Formatted at 2026-08-09 20:22:20 on dist-test-slave-6k22"
server_key: "cb5bfeaf63098f9283427d6946b77d20"
server_key_iv: "43c14c62a33f477a59a12aa1b29ea4e3"
server_key_version: "encryptionkey@0"
I20260809 20:22:20.256516 27639 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 20:22:20.259734 27657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:20.260969 27639 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 20:22:20.261162 27639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "1dad334386314e3ea7a8736a9d312136"
format_stamp: "Formatted at 2026-08-09 20:22:20 on dist-test-slave-6k22"
server_key: "cb5bfeaf63098f9283427d6946b77d20"
server_key_iv: "43c14c62a33f477a59a12aa1b29ea4e3"
server_key_version: "encryptionkey@0"
I20260809 20:22:20.261381 27639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:20.282517 27639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:20.286990 27639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:20.287328 27639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:20.288420 27639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:22:20.289958 27639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:22:20.290077 27639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:20.290256 27639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:22:20.290339 27639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:20.328258 27639 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:38275
I20260809 20:22:20.328336 27794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:38275 every 8 connection(s)
I20260809 20:22:20.329746 27639 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:22:20.337445 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 27639
I20260809 20:22:20.337643 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:22:20.338172 5364 external_mini_cluster.cc:1428] Setting key e171d4854923a5b8a96857436c9d570a
Aug 09 20:22:20 dist-test-slave-6k22 krb5kdc[27509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306940, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.1@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:22:20.351967 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:20.332480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:37051 (local address 127.5.61.62:36215)
0809 20:22:20.332990 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:20.332995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:20.333968 (+ 973us) server_negotiation.cc:407] Connection header received
0809 20:22:20.335863 (+ 1895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:20.335868 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:20.335974 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:20.336099 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:20.337640 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:20.338731 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:20.340830 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:20.341545 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:20.341688 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:20.345135 (+ 3447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:20.345143 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:20.345151 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:20.345195 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:20.348691 (+ 3496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:20.349414 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:20.349423 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:20.349429 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:20.349598 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:20.350030 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:20.350037 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:20.350043 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:20.350413 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:20.350567 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:20.351233 (+ 666us) server_negotiation.cc:299] Negotiation successful
0809 20:22:20.351484 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":274,"threads_started":1}
I20260809 20:22:20.352734 27799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:20.333070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:36215 (local address 127.5.61.1:37051)
0809 20:22:20.333805 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:20.333847 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:20.335546 (+ 1699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:20.336361 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:20.336370 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:20.336885 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:22:20.337429 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:20.337445 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:20.338946 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:20.338953 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:20.340673 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:20.340685 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:20.341753 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:20.341758 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:20.341985 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:20.342714 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:22:20.342734 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:22:20.344922 (+ 2188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:22:20.348848 (+ 3926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:22:20.348854 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:22:20.348875 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:22:20.349280 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:22:20.349708 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:22:20.349711 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:22:20.349713 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:22:20.349912 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:22:20.350533 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:22:20.350538 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:22:20.350832 (+ 294us) client_negotiation.cc:769] Sending connection context
0809 20:22:20.351059 (+ 227us) client_negotiation.cc:240] Negotiation successful
0809 20:22:20.351428 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":286,"threads_started":1}
I20260809 20:22:20.354308 27795 heartbeater.cc:347] Connected to a master server at 127.5.61.62:36215
I20260809 20:22:20.354632 27795 heartbeater.cc:464] Registering TS with master...
I20260809 20:22:20.355500 27795 heartbeater.cc:511] Master 127.5.61.62:36215 requested a full tablet report, sending...
I20260809 20:22:20.357712 27568 ts_manager.cc:194] Registered new tserver with Master: 1dad334386314e3ea7a8736a9d312136 (127.5.61.1:38275)
I20260809 20:22:20.359164 27568 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.1@KRBTEST.COM'} at 127.5.61.1:37051
WARNING: no policy specified for kudu/127.5.61.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
Entry for principal kudu/127.5.61.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab.
I20260809 20:22:20.406821 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:36215
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:41233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:20.755401 27803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:20.755735 27803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:20.755927 27803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:20.762956 27803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:20.763088 27803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:20.763273 27803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:22:20.772588 27803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:41233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.2.keytab
--principal=kudu/127.5.61.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:36215
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:20.773833 27803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:20.775115 27803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:20.787878 27808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:20.788103 27810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:20.787907 27813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:20.789582 27803 server_base.cc:1034] running on GCE node
I20260809 20:22:20.790130 27803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:20.790948 27803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:20.792155 27803 hybrid_clock.cc:648] HybridClock initialized: now 1786306940792112 us; error 56 us; skew 500 ppm
I20260809 20:22:20.792439 27803 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:20 dist-test-slave-6k22 krb5kdc[27509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306940, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:20.795491 27803 init.cc:377] Logged in from keytab as kudu/127.5.61.2@KRBTEST.COM (short username kudu)
I20260809 20:22:20.796902 27803 webserver.cc:460] Webserver started at http://127.5.61.2:35815/ using document root <none> and password file <none>
I20260809 20:22:20.797554 27803 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:20.797706 27803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:20.798007 27803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:20.800889 27803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "b36031d516fa4cb68072050ad948310b"
format_stamp: "Formatted at 2026-08-09 20:22:20 on dist-test-slave-6k22"
server_key: "b760d6e69f54c3bd571a6ef9019284e2"
server_key_iv: "91fb14e53129c5d84b074a1a8bc18c71"
server_key_version: "encryptionkey@0"
I20260809 20:22:20.801726 27803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "b36031d516fa4cb68072050ad948310b"
format_stamp: "Formatted at 2026-08-09 20:22:20 on dist-test-slave-6k22"
server_key: "b760d6e69f54c3bd571a6ef9019284e2"
server_key_iv: "91fb14e53129c5d84b074a1a8bc18c71"
server_key_version: "encryptionkey@0"
I20260809 20:22:20.806670 27803 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 20:22:20.810211 27823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:20.811509 27803 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 20:22:20.811738 27803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "b36031d516fa4cb68072050ad948310b"
format_stamp: "Formatted at 2026-08-09 20:22:20 on dist-test-slave-6k22"
server_key: "b760d6e69f54c3bd571a6ef9019284e2"
server_key_iv: "91fb14e53129c5d84b074a1a8bc18c71"
server_key_version: "encryptionkey@0"
I20260809 20:22:20.811928 27803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:20.823817 27803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:20.829044 27803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:20.829429 27803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:20.830775 27803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:22:20.835129 27803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:22:20.835243 27803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:20.835377 27803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:22:20.835489 27803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:20.873581 27803 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:43137
I20260809 20:22:20.873590 27965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:43137 every 8 connection(s)
I20260809 20:22:20.875484 27803 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:22:20.876880 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 27803
I20260809 20:22:20.877063 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:22:20.877485 5364 external_mini_cluster.cc:1428] Setting key 9d4afcccb57ee9977d3044d32bb8aec8
Aug 09 20:22:20 dist-test-slave-6k22 krb5kdc[27509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306940, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.2@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:22:20.901121 27969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:20.881017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:36289 (local address 127.5.61.62:36215)
0809 20:22:20.886914 (+ 5897us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:20.886919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:20.886934 (+ 15us) server_negotiation.cc:407] Connection header received
0809 20:22:20.886982 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:20.886985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:20.887047 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:20.887178 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:20.888796 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:20.889516 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:20.891006 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:20.891461 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:20.891606 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:20.895478 (+ 3872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:20.895484 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:20.895494 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:20.895525 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:20.898108 (+ 2583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:20.898824 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:20.898829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:20.898832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:20.898962 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:20.899391 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:20.899395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:20.899399 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:20.899745 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:20.899893 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:20.900517 (+ 624us) server_negotiation.cc:299] Negotiation successful
0809 20:22:20.900744 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5800,"thread_start_us":5708,"threads_started":1}
I20260809 20:22:20.902045 27968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:20.880437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:36215 (local address 127.5.61.2:36289)
0809 20:22:20.881398 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:20.881463 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:20.884385 (+ 2922us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:20.887429 (+ 3044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:20.887438 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:20.888065 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:22:20.888626 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:20.888644 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:20.889662 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:20.889668 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:20.890855 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:20.890866 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:20.891717 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:20.891724 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:20.891950 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:20.892639 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:22:20.892664 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:22:20.895254 (+ 2590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:22:20.898245 (+ 2991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:22:20.898254 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:22:20.898268 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:22:20.898698 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:22:20.899074 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:22:20.899078 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:22:20.899082 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:22:20.899272 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:22:20.899870 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:22:20.899877 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:22:20.900180 (+ 303us) client_negotiation.cc:769] Sending connection context
0809 20:22:20.900421 (+ 241us) client_negotiation.cc:240] Negotiation successful
0809 20:22:20.900906 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":614,"threads_started":1}
I20260809 20:22:20.903391 27966 heartbeater.cc:347] Connected to a master server at 127.5.61.62:36215
I20260809 20:22:20.903769 27966 heartbeater.cc:464] Registering TS with master...
I20260809 20:22:20.904628 27966 heartbeater.cc:511] Master 127.5.61.62:36215 requested a full tablet report, sending...
I20260809 20:22:20.906216 27568 ts_manager.cc:194] Registered new tserver with Master: b36031d516fa4cb68072050ad948310b (127.5.61.2:43137)
I20260809 20:22:20.907049 27568 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.2@KRBTEST.COM'} at 127.5.61.2:36289
WARNING: no policy specified for kudu/127.5.61.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
Entry for principal kudu/127.5.61.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab.
I20260809 20:22:20.946223 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:36215
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:41233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:21.281440 27974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:21.281698 27974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:21.281858 27974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:21.288965 27974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:21.289083 27974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:21.289296 27974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:22:21.298709 27974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:41233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.3.keytab
--principal=kudu/127.5.61.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:36215
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:21.300000 27974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:21.301298 27974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:21.314085 27980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:21.314141 27983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:21.314170 27981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:21.315280 27974 server_base.cc:1034] running on GCE node
I20260809 20:22:21.316056 27974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:21.316946 27974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:21.318169 27974 hybrid_clock.cc:648] HybridClock initialized: now 1786306941318125 us; error 53 us; skew 500 ppm
I20260809 20:22:21.318507 27974 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:21 dist-test-slave-6k22 krb5kdc[27509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306941, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:21.321552 27974 init.cc:377] Logged in from keytab as kudu/127.5.61.3@KRBTEST.COM (short username kudu)
I20260809 20:22:21.322866 27974 webserver.cc:460] Webserver started at http://127.5.61.3:33643/ using document root <none> and password file <none>
I20260809 20:22:21.323477 27974 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:21.323630 27974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:21.323979 27974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:21.326992 27974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "cc499e5f479643d0b323f535a956118f"
format_stamp: "Formatted at 2026-08-09 20:22:21 on dist-test-slave-6k22"
server_key: "43d5ca5f0f2363682dcf06983e268368"
server_key_iv: "1380f6ffa5557a06dea5e1f79ac58e88"
server_key_version: "encryptionkey@0"
I20260809 20:22:21.327941 27974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "cc499e5f479643d0b323f535a956118f"
format_stamp: "Formatted at 2026-08-09 20:22:21 on dist-test-slave-6k22"
server_key: "43d5ca5f0f2363682dcf06983e268368"
server_key_iv: "1380f6ffa5557a06dea5e1f79ac58e88"
server_key_version: "encryptionkey@0"
I20260809 20:22:21.333093 27974 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 20:22:21.336943 27992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:21.338188 27974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 20:22:21.338359 27974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "cc499e5f479643d0b323f535a956118f"
format_stamp: "Formatted at 2026-08-09 20:22:21 on dist-test-slave-6k22"
server_key: "43d5ca5f0f2363682dcf06983e268368"
server_key_iv: "1380f6ffa5557a06dea5e1f79ac58e88"
server_key_version: "encryptionkey@0"
I20260809 20:22:21.338581 27974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:21.362857 27795 heartbeater.cc:503] Master 127.5.61.62:36215 was elected leader, sending a full tablet report...
I20260809 20:22:21.373449 27974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:21.377890 27974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:21.378201 27974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:21.379209 27974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:22:21.380617 27974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:22:21.380720 27974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:21.380889 27974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:22:21.380960 27974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:21.416074 27974 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:35463
I20260809 20:22:21.416107 28126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:35463 every 8 connection(s)
I20260809 20:22:21.417618 27974 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:22:21.423739 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 27974
I20260809 20:22:21.423962 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:22:21.424450 5364 external_mini_cluster.cc:1428] Setting key 69ffe0752509494207e52cb2140ca942
Aug 09 20:22:21 dist-test-slave-6k22 krb5kdc[27509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306941, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.3@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:22:21.440331 28132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:21.420989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:44225 (local address 127.5.61.62:36215)
0809 20:22:21.421493 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:21.421498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:21.422390 (+ 892us) server_negotiation.cc:407] Connection header received
0809 20:22:21.424496 (+ 2106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:21.424502 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:21.424601 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:21.424726 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:21.426346 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.427233 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.429129 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.429772 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.429949 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:21.433784 (+ 3835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:21.433790 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:21.433793 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:21.433829 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:21.436859 (+ 3030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:21.437736 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:21.437743 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:21.437746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:21.437875 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:21.438346 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:21.438352 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:21.438360 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:21.438660 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:21.438829 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:21.439737 (+ 908us) server_negotiation.cc:299] Negotiation successful
0809 20:22:21.439992 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":285,"threads_started":1}
I20260809 20:22:21.441311 28133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:21.421508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:36215 (local address 127.5.61.3:44225)
0809 20:22:21.422221 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:21.422261 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:21.424215 (+ 1954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:21.424988 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:21.424996 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:21.425599 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:22:21.426156 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:21.426172 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.427382 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.427392 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:21.428980 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:21.428992 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.430041 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.430049 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:21.430296 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:21.430953 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:22:21.430980 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:22:21.433554 (+ 2574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:22:21.437011 (+ 3457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:22:21.437020 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:22:21.437035 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:22:21.437488 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:22:21.437995 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:22:21.438000 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:22:21.438004 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:22:21.438218 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:22:21.438820 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:22:21.438828 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:22:21.439269 (+ 441us) client_negotiation.cc:769] Sending connection context
0809 20:22:21.439630 (+ 361us) client_negotiation.cc:240] Negotiation successful
0809 20:22:21.440170 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":295,"threads_started":1}
I20260809 20:22:21.442701 28130 heartbeater.cc:347] Connected to a master server at 127.5.61.62:36215
I20260809 20:22:21.443017 28130 heartbeater.cc:464] Registering TS with master...
I20260809 20:22:21.443768 28130 heartbeater.cc:511] Master 127.5.61.62:36215 requested a full tablet report, sending...
I20260809 20:22:21.445128 27568 ts_manager.cc:194] Registered new tserver with Master: cc499e5f479643d0b323f535a956118f (127.5.61.3:35463)
I20260809 20:22:21.445891 27568 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.61.3@KRBTEST.COM'} at 127.5.61.3:44225
I20260809 20:22:21.452082 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 20:22:21.468724 28132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:21.456443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54968 (local address 127.5.61.62:36215)
0809 20:22:21.456662 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:21.456670 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:21.457037 (+ 367us) server_negotiation.cc:407] Connection header received
0809 20:22:21.457190 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:21.457194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:21.457260 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:21.457337 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:21.458823 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.459508 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.460917 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.461308 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.461397 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:21.463760 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:21.463766 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:21.463769 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:21.463798 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:21.466271 (+ 2473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:21.466956 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:21.466959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:21.466961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:21.467071 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:21.467467 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:21.467470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:21.467471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:21.467774 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:21.467948 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:21.468226 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 20:22:21.468399 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 20:22:21.472298 27568 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54968:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:22:21.479102 27568 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:22:21.512380 28147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:21.492850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:43137 (local address 127.0.0.1:55858)
0809 20:22:21.499451 (+ 6601us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:21.499475 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:21.499646 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:21.505214 (+ 5568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:21.505221 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:21.505249 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:21.505710 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:21.505732 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.507434 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.507439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:21.510004 (+ 2565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:21.510016 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.511307 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.511314 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:21.511509 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:21.511582 (+ 73us) client_negotiation.cc:769] Sending connection context
0809 20:22:21.511723 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 20:22:21.511912 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6437,"mutex_wait_us":185,"thread_start_us":6250,"threads_started":1}
I20260809 20:22:21.513375 28148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:21.493154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55858 (local address 127.5.61.2:43137)
0809 20:22:21.504552 (+ 11398us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:21.504565 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:21.504600 (+ 35us) server_negotiation.cc:407] Connection header received
0809 20:22:21.504701 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:21.504708 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:21.504987 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:21.505211 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:21.505948 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.507285 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.510166 (+ 2881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.511121 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.511272 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:21.511468 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:21.511782 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 20:22:21.513076 (+ 1294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11167,"thread_start_us":11002,"threads_started":1}
I20260809 20:22:21.513614 28143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:21.491416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:35463 (local address 127.0.0.1:49162)
0809 20:22:21.492371 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:21.492404 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:21.492599 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:21.505409 (+ 12810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:21.505414 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:21.505437 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:21.506040 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:21.506053 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.508275 (+ 2222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.508293 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:21.511012 (+ 2719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:21.511027 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.512833 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.512839 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:21.512983 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:21.513011 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 20:22:21.513078 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 20:22:21.513209 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":855,"mutex_wait_us":210,"spinlock_wait_cycles":11136,"thread_start_us":468,"threads_started":1}
I20260809 20:22:21.513952 28145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:21.491788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51872 (local address 127.5.61.1:38275)
0809 20:22:21.504731 (+ 12943us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:21.504750 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:21.504778 (+ 28us) server_negotiation.cc:407] Connection header received
0809 20:22:21.504878 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:21.504885 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:21.505139 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:21.505330 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:21.506487 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.508108 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.511198 (+ 3090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.512690 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.512804 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:21.513027 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:21.513446 (+ 419us) server_negotiation.cc:299] Negotiation successful
0809 20:22:21.513588 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12757,"thread_start_us":12984,"threads_started":1}
I20260809 20:22:21.513993 28144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:21.491176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:38275 (local address 127.0.0.1:51872)
0809 20:22:21.492351 (+ 1175us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:21.492405 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:21.492599 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:21.505596 (+ 12997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:21.505600 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:21.505635 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:21.506235 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:21.506249 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.508277 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.508293 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:21.511012 (+ 2719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:21.511027 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.513155 (+ 2128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.513159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:21.513261 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:21.513280 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 20:22:21.513406 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 20:22:21.513524 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":885,"mutex_wait_us":134,"spinlock_wait_cycles":9088,"thread_start_us":1829,"threads_started":1}
I20260809 20:22:21.514812 28146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:21.492497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49162 (local address 127.5.61.3:35463)
0809 20:22:21.504898 (+ 12401us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:21.504905 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:21.504933 (+ 28us) server_negotiation.cc:407] Connection header received
0809 20:22:21.505032 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:21.505037 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:21.505200 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:21.505576 (+ 376us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:21.506249 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.508121 (+ 1872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.511180 (+ 3059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.512616 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.514004 (+ 1388us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:21.514247 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:21.514318 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 20:22:21.514474 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12241,"thread_start_us":12079,"threads_started":1}
I20260809 20:22:21.517405 27882 tablet_service.cc:1467] Processing CreateTablet for tablet 45e73fd01b0e4f1dbd40a838cb46d51e (DEFAULT_TABLE table=test-table [id=3bf2e010943645f7a8720a73e77799d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:22:21.517496 27711 tablet_service.cc:1467] Processing CreateTablet for tablet 45e73fd01b0e4f1dbd40a838cb46d51e (DEFAULT_TABLE table=test-table [id=3bf2e010943645f7a8720a73e77799d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:22:21.519114 27882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45e73fd01b0e4f1dbd40a838cb46d51e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:21.519134 27711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45e73fd01b0e4f1dbd40a838cb46d51e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:21.518859 28054 tablet_service.cc:1467] Processing CreateTablet for tablet 45e73fd01b0e4f1dbd40a838cb46d51e (DEFAULT_TABLE table=test-table [id=3bf2e010943645f7a8720a73e77799d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:22:21.520195 28054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45e73fd01b0e4f1dbd40a838cb46d51e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:21.530357 28149 tablet_bootstrap.cc:492] T 45e73fd01b0e4f1dbd40a838cb46d51e P b36031d516fa4cb68072050ad948310b: Bootstrap starting.
I20260809 20:22:21.534114 28149 tablet_bootstrap.cc:654] T 45e73fd01b0e4f1dbd40a838cb46d51e P b36031d516fa4cb68072050ad948310b: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:21.534863 28150 tablet_bootstrap.cc:492] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136: Bootstrap starting.
I20260809 20:22:21.535534 28151 tablet_bootstrap.cc:492] T 45e73fd01b0e4f1dbd40a838cb46d51e P cc499e5f479643d0b323f535a956118f: Bootstrap starting.
I20260809 20:22:21.535610 28149 log.cc:824] T 45e73fd01b0e4f1dbd40a838cb46d51e P b36031d516fa4cb68072050ad948310b: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:21.538204 28151 tablet_bootstrap.cc:654] T 45e73fd01b0e4f1dbd40a838cb46d51e P cc499e5f479643d0b323f535a956118f: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:21.538811 28150 tablet_bootstrap.cc:654] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:21.539254 28151 log.cc:824] T 45e73fd01b0e4f1dbd40a838cb46d51e P cc499e5f479643d0b323f535a956118f: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:21.540243 28150 log.cc:824] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:21.542315 28149 tablet_bootstrap.cc:492] T 45e73fd01b0e4f1dbd40a838cb46d51e P b36031d516fa4cb68072050ad948310b: No bootstrap required, opened a new log
I20260809 20:22:21.542330 28151 tablet_bootstrap.cc:492] T 45e73fd01b0e4f1dbd40a838cb46d51e P cc499e5f479643d0b323f535a956118f: No bootstrap required, opened a new log
I20260809 20:22:21.542573 28149 ts_tablet_manager.cc:1397] T 45e73fd01b0e4f1dbd40a838cb46d51e P b36031d516fa4cb68072050ad948310b: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260809 20:22:21.542593 28151 ts_tablet_manager.cc:1397] T 45e73fd01b0e4f1dbd40a838cb46d51e P cc499e5f479643d0b323f535a956118f: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260809 20:22:21.543140 28150 tablet_bootstrap.cc:492] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136: No bootstrap required, opened a new log
I20260809 20:22:21.543396 28150 ts_tablet_manager.cc:1397] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260809 20:22:21.550109 28149 raft_consensus.cc:348] T 45e73fd01b0e4f1dbd40a838cb46d51e P b36031d516fa4cb68072050ad948310b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dad334386314e3ea7a8736a9d312136" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 38275 } } peers { permanent_uuid: "cc499e5f479643d0b323f535a956118f" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 35463 } } peers { permanent_uuid: "b36031d516fa4cb68072050ad948310b" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 43137 } }
I20260809 20:22:21.550469 28149 raft_consensus.cc:374] T 45e73fd01b0e4f1dbd40a838cb46d51e P b36031d516fa4cb68072050ad948310b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:21.550583 28149 raft_consensus.cc:729] T 45e73fd01b0e4f1dbd40a838cb46d51e P b36031d516fa4cb68072050ad948310b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b36031d516fa4cb68072050ad948310b, State: Initialized, Role: FOLLOWER
I20260809 20:22:21.551198 28149 consensus_queue.cc:260] T 45e73fd01b0e4f1dbd40a838cb46d51e P b36031d516fa4cb68072050ad948310b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dad334386314e3ea7a8736a9d312136" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 38275 } } peers { permanent_uuid: "cc499e5f479643d0b323f535a956118f" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 35463 } } peers { permanent_uuid: "b36031d516fa4cb68072050ad948310b" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 43137 } }
I20260809 20:22:21.552336 27966 heartbeater.cc:503] Master 127.5.61.62:36215 was elected leader, sending a full tablet report...
I20260809 20:22:21.553337 28151 raft_consensus.cc:348] T 45e73fd01b0e4f1dbd40a838cb46d51e P cc499e5f479643d0b323f535a956118f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dad334386314e3ea7a8736a9d312136" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 38275 } } peers { permanent_uuid: "cc499e5f479643d0b323f535a956118f" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 35463 } } peers { permanent_uuid: "b36031d516fa4cb68072050ad948310b" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 43137 } }
I20260809 20:22:21.553794 28151 raft_consensus.cc:374] T 45e73fd01b0e4f1dbd40a838cb46d51e P cc499e5f479643d0b323f535a956118f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:21.553905 28151 raft_consensus.cc:729] T 45e73fd01b0e4f1dbd40a838cb46d51e P cc499e5f479643d0b323f535a956118f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc499e5f479643d0b323f535a956118f, State: Initialized, Role: FOLLOWER
I20260809 20:22:21.554293 28149 ts_tablet_manager.cc:1428] T 45e73fd01b0e4f1dbd40a838cb46d51e P b36031d516fa4cb68072050ad948310b: Time spent starting tablet: real 0.012s user 0.005s sys 0.008s
I20260809 20:22:21.554100 28150 raft_consensus.cc:348] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dad334386314e3ea7a8736a9d312136" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 38275 } } peers { permanent_uuid: "cc499e5f479643d0b323f535a956118f" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 35463 } } peers { permanent_uuid: "b36031d516fa4cb68072050ad948310b" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 43137 } }
I20260809 20:22:21.554495 28150 raft_consensus.cc:374] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:21.554567 28151 consensus_queue.cc:260] T 45e73fd01b0e4f1dbd40a838cb46d51e P cc499e5f479643d0b323f535a956118f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dad334386314e3ea7a8736a9d312136" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 38275 } } peers { permanent_uuid: "cc499e5f479643d0b323f535a956118f" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 35463 } } peers { permanent_uuid: "b36031d516fa4cb68072050ad948310b" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 43137 } }
I20260809 20:22:21.554790 28150 raft_consensus.cc:729] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dad334386314e3ea7a8736a9d312136, State: Initialized, Role: FOLLOWER
I20260809 20:22:21.555403 28150 consensus_queue.cc:260] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dad334386314e3ea7a8736a9d312136" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 38275 } } peers { permanent_uuid: "cc499e5f479643d0b323f535a956118f" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 35463 } } peers { permanent_uuid: "b36031d516fa4cb68072050ad948310b" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 43137 } }
I20260809 20:22:21.555761 28130 heartbeater.cc:503] Master 127.5.61.62:36215 was elected leader, sending a full tablet report...
I20260809 20:22:21.556464 28151 ts_tablet_manager.cc:1428] T 45e73fd01b0e4f1dbd40a838cb46d51e P cc499e5f479643d0b323f535a956118f: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260809 20:22:21.558643 28150 ts_tablet_manager.cc:1428] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
W20260809 20:22:21.582304 27797 tablet.cc:2369] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 20:22:21.630270 27967 tablet.cc:2369] T 45e73fd01b0e4f1dbd40a838cb46d51e P b36031d516fa4cb68072050ad948310b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 20:22:21.669764 28131 tablet.cc:2369] T 45e73fd01b0e4f1dbd40a838cb46d51e P cc499e5f479643d0b323f535a956118f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:22:21.748916 28162 raft_consensus.cc:482] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:22:21.749215 28162 raft_consensus.cc:504] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dad334386314e3ea7a8736a9d312136" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 38275 } } peers { permanent_uuid: "cc499e5f479643d0b323f535a956118f" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 35463 } } peers { permanent_uuid: "b36031d516fa4cb68072050ad948310b" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 43137 } }
I20260809 20:22:21.750944 28162 leader_election.cc:288] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc499e5f479643d0b323f535a956118f (127.5.61.3:35463), b36031d516fa4cb68072050ad948310b (127.5.61.2:43137)
I20260809 20:22:21.758437 28146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:21.751512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:34011 (local address 127.5.61.3:35463)
0809 20:22:21.751725 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:21.751729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:21.752323 (+ 594us) server_negotiation.cc:407] Connection header received
0809 20:22:21.752599 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:21.752605 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:21.752687 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:21.752792 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:21.753570 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.754445 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.756431 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.757483 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.757579 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:21.757652 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:21.758059 (+ 407us) server_negotiation.cc:299] Negotiation successful
0809 20:22:21.758146 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 20:22:21.758649 28163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:21.751437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:35463 (local address 127.5.61.1:34011)
0809 20:22:21.752205 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:21.752226 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:21.752375 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:21.752900 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:21.752905 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:21.752942 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:21.753355 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:21.753365 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.754595 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.754599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:21.756278 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:21.756290 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.757687 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.757693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:21.757886 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:21.757921 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 20:22:21.758019 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 20:22:21.758147 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"mutex_wait_us":20,"thread_start_us":462,"threads_started":1}
I20260809 20:22:21.759274 28148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:21.752560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:48639 (local address 127.5.61.2:43137)
0809 20:22:21.752798 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:21.752804 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:21.753261 (+ 457us) server_negotiation.cc:407] Connection header received
0809 20:22:21.753489 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:21.753494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:21.753562 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:21.753660 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:21.754230 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.755074 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.757142 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.758157 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.758297 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:21.758414 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:21.758880 (+ 466us) server_negotiation.cc:299] Negotiation successful
0809 20:22:21.759029 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 20:22:21.759749 28077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45e73fd01b0e4f1dbd40a838cb46d51e" candidate_uuid: "1dad334386314e3ea7a8736a9d312136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc499e5f479643d0b323f535a956118f" is_pre_election: true
I20260809 20:22:21.760036 28164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:21.752370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:43137 (local address 127.5.61.1:48639)
0809 20:22:21.753156 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:21.753170 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:21.753318 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:21.753682 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:21.753686 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:21.753711 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:21.754043 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:21.754051 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.755249 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.755253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:21.756981 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:21.756996 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.758414 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.758422 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:21.758616 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:21.758653 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 20:22:21.759529 (+ 876us) client_negotiation.cc:240] Negotiation successful
0809 20:22:21.759678 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"mutex_wait_us":22,"thread_start_us":357,"threads_started":1}
I20260809 20:22:21.760365 28077 raft_consensus.cc:2454] T 45e73fd01b0e4f1dbd40a838cb46d51e P cc499e5f479643d0b323f535a956118f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dad334386314e3ea7a8736a9d312136 in term 0.
I20260809 20:22:21.761001 27660 leader_election.cc:302] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dad334386314e3ea7a8736a9d312136, cc499e5f479643d0b323f535a956118f; no voters:
I20260809 20:22:21.760965 27908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45e73fd01b0e4f1dbd40a838cb46d51e" candidate_uuid: "1dad334386314e3ea7a8736a9d312136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b36031d516fa4cb68072050ad948310b" is_pre_election: true
I20260809 20:22:21.761379 28162 raft_consensus.cc:2781] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:22:21.761557 28162 raft_consensus.cc:482] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:22:21.761503 27908 raft_consensus.cc:2454] T 45e73fd01b0e4f1dbd40a838cb46d51e P b36031d516fa4cb68072050ad948310b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dad334386314e3ea7a8736a9d312136 in term 0.
I20260809 20:22:21.761667 28162 raft_consensus.cc:3037] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:21.764002 28162 raft_consensus.cc:504] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dad334386314e3ea7a8736a9d312136" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 38275 } } peers { permanent_uuid: "cc499e5f479643d0b323f535a956118f" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 35463 } } peers { permanent_uuid: "b36031d516fa4cb68072050ad948310b" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 43137 } }
I20260809 20:22:21.764510 28162 leader_election.cc:288] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136 [CANDIDATE]: Term 1 election: Requested vote from peers cc499e5f479643d0b323f535a956118f (127.5.61.3:35463), b36031d516fa4cb68072050ad948310b (127.5.61.2:43137)
I20260809 20:22:21.764932 28077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45e73fd01b0e4f1dbd40a838cb46d51e" candidate_uuid: "1dad334386314e3ea7a8736a9d312136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc499e5f479643d0b323f535a956118f"
I20260809 20:22:21.765027 27908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45e73fd01b0e4f1dbd40a838cb46d51e" candidate_uuid: "1dad334386314e3ea7a8736a9d312136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b36031d516fa4cb68072050ad948310b"
I20260809 20:22:21.765170 28077 raft_consensus.cc:3037] T 45e73fd01b0e4f1dbd40a838cb46d51e P cc499e5f479643d0b323f535a956118f [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:21.765219 27908 raft_consensus.cc:3037] T 45e73fd01b0e4f1dbd40a838cb46d51e P b36031d516fa4cb68072050ad948310b [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:21.768458 27908 raft_consensus.cc:2454] T 45e73fd01b0e4f1dbd40a838cb46d51e P b36031d516fa4cb68072050ad948310b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dad334386314e3ea7a8736a9d312136 in term 1.
I20260809 20:22:21.768463 28077 raft_consensus.cc:2454] T 45e73fd01b0e4f1dbd40a838cb46d51e P cc499e5f479643d0b323f535a956118f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dad334386314e3ea7a8736a9d312136 in term 1.
I20260809 20:22:21.768853 27660 leader_election.cc:302] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dad334386314e3ea7a8736a9d312136, cc499e5f479643d0b323f535a956118f; no voters:
I20260809 20:22:21.769151 28162 raft_consensus.cc:2781] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:21.769555 28162 raft_consensus.cc:686] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136 [term 1 LEADER]: Becoming Leader. State: Replica: 1dad334386314e3ea7a8736a9d312136, State: Running, Role: LEADER
I20260809 20:22:21.770099 28162 consensus_queue.cc:237] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dad334386314e3ea7a8736a9d312136" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 38275 } } peers { permanent_uuid: "cc499e5f479643d0b323f535a956118f" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 35463 } } peers { permanent_uuid: "b36031d516fa4cb68072050ad948310b" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 43137 } }
I20260809 20:22:21.774502 27567 catalog_manager.cc:5495] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dad334386314e3ea7a8736a9d312136 (127.5.61.1). New cstate: current_term: 1 leader_uuid: "1dad334386314e3ea7a8736a9d312136" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dad334386314e3ea7a8736a9d312136" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 38275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc499e5f479643d0b323f535a956118f" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 35463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b36031d516fa4cb68072050ad948310b" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 43137 } health_report { overall_health: UNKNOWN } } }
I20260809 20:22:21.798866 28145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:21.793998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51882 (local address 127.5.61.1:38275)
0809 20:22:21.794172 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:21.794176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:21.794192 (+ 16us) server_negotiation.cc:407] Connection header received
0809 20:22:21.794263 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:21.794266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:21.794351 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:21.794477 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:22:21.794783 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.795488 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.796941 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.797336 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.797436 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:21.797878 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:22:21.798440 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:22:21.798547 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:21.798620 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 20:22:21.798681 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 20:22:21.806456 27908 raft_consensus.cc:1261] T 45e73fd01b0e4f1dbd40a838cb46d51e P b36031d516fa4cb68072050ad948310b [term 1 FOLLOWER]: Refusing update from remote peer 1dad334386314e3ea7a8736a9d312136: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:22:21.806499 28077 raft_consensus.cc:1261] T 45e73fd01b0e4f1dbd40a838cb46d51e P cc499e5f479643d0b323f535a956118f [term 1 FOLLOWER]: Refusing update from remote peer 1dad334386314e3ea7a8736a9d312136: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:22:21.807354 28165 consensus_queue.cc:1035] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b36031d516fa4cb68072050ad948310b" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 43137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:22:21.807792 28162 consensus_queue.cc:1035] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc499e5f479643d0b323f535a956118f" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 35463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:22:21.820829 28170 mvcc.cc:205] Tried to move back new op lower bound from 7316713233630035968 to 7316713233493794816. Current Snapshot: MvccSnapshot[applied={T|T < 7316713233630035968}]
I20260809 20:22:21.831875 27567 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:22:21.832131 27567 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:22:21.835778 27567 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42ef4160d4cf47379fe6b4faa9b7cbf9: Sending DeleteTablet for 3 replicas of tablet 45e73fd01b0e4f1dbd40a838cb46d51e
I20260809 20:22:21.837193 27711 tablet_service.cc:1514] Processing DeleteTablet for tablet 45e73fd01b0e4f1dbd40a838cb46d51e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:22:21 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:51872
I20260809 20:22:21.837352 27882 tablet_service.cc:1514] Processing DeleteTablet for tablet 45e73fd01b0e4f1dbd40a838cb46d51e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:22:21 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:55858
I20260809 20:22:21.837918 28054 tablet_service.cc:1514] Processing DeleteTablet for tablet 45e73fd01b0e4f1dbd40a838cb46d51e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:22:21 UTC) from {username='kudu', principal='kudu/127.5.61.62@KRBTEST.COM'} at 127.0.0.1:49162
I20260809 20:22:21.838157 28179 tablet_replica.cc:330] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136: stopping tablet replica
I20260809 20:22:21.838649 28179 raft_consensus.cc:2229] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136 [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:22:21.839354 28179 raft_consensus.cc:2258] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:22:21.839700 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 27639
I20260809 20:22:21.849097 28179 ts_tablet_manager.cc:1905] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:22:21.851114 28182 tablet_replica.cc:330] T 45e73fd01b0e4f1dbd40a838cb46d51e P cc499e5f479643d0b323f535a956118f: stopping tablet replica
I20260809 20:22:21.851708 28182 raft_consensus.cc:2229] T 45e73fd01b0e4f1dbd40a838cb46d51e P cc499e5f479643d0b323f535a956118f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:22:21.852298 28182 raft_consensus.cc:2258] T 45e73fd01b0e4f1dbd40a838cb46d51e P cc499e5f479643d0b323f535a956118f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:22:21.853345 28180 tablet_replica.cc:330] T 45e73fd01b0e4f1dbd40a838cb46d51e P b36031d516fa4cb68072050ad948310b: stopping tablet replica
I20260809 20:22:21.853713 28180 raft_consensus.cc:2229] T 45e73fd01b0e4f1dbd40a838cb46d51e P b36031d516fa4cb68072050ad948310b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:22:21.854228 28180 raft_consensus.cc:2258] T 45e73fd01b0e4f1dbd40a838cb46d51e P b36031d516fa4cb68072050ad948310b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:22:21.854630 28182 ts_tablet_manager.cc:1905] T 45e73fd01b0e4f1dbd40a838cb46d51e P cc499e5f479643d0b323f535a956118f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:22:21.860790 28180 ts_tablet_manager.cc:1905] T 45e73fd01b0e4f1dbd40a838cb46d51e P b36031d516fa4cb68072050ad948310b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:22:21.860857 28179 ts_tablet_manager.cc:1918] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:22:21.861018 28179 log.cc:1196] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/45e73fd01b0e4f1dbd40a838cb46d51e
I20260809 20:22:21.861605 28179 ts_tablet_manager.cc:1939] T 45e73fd01b0e4f1dbd40a838cb46d51e P 1dad334386314e3ea7a8736a9d312136: Deleting consensus metadata
I20260809 20:22:21.863405 27551 catalog_manager.cc:4841] TS 1dad334386314e3ea7a8736a9d312136 (127.5.61.1:38275): tablet 45e73fd01b0e4f1dbd40a838cb46d51e (table test-table [id=3bf2e010943645f7a8720a73e77799d1]) successfully deleted
I20260809 20:22:21.866048 28182 ts_tablet_manager.cc:1918] T 45e73fd01b0e4f1dbd40a838cb46d51e P cc499e5f479643d0b323f535a956118f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:22:21.866211 28182 log.cc:1196] T 45e73fd01b0e4f1dbd40a838cb46d51e P cc499e5f479643d0b323f535a956118f: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/45e73fd01b0e4f1dbd40a838cb46d51e
I20260809 20:22:21.866766 28182 ts_tablet_manager.cc:1939] T 45e73fd01b0e4f1dbd40a838cb46d51e P cc499e5f479643d0b323f535a956118f: Deleting consensus metadata
I20260809 20:22:21.866930 28180 ts_tablet_manager.cc:1918] T 45e73fd01b0e4f1dbd40a838cb46d51e P b36031d516fa4cb68072050ad948310b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:22:21.867101 28180 log.cc:1196] T 45e73fd01b0e4f1dbd40a838cb46d51e P b36031d516fa4cb68072050ad948310b: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/45e73fd01b0e4f1dbd40a838cb46d51e
I20260809 20:22:21.867604 28180 ts_tablet_manager.cc:1939] T 45e73fd01b0e4f1dbd40a838cb46d51e P b36031d516fa4cb68072050ad948310b: Deleting consensus metadata
I20260809 20:22:21.868608 27549 catalog_manager.cc:4841] TS cc499e5f479643d0b323f535a956118f (127.5.61.3:35463): tablet 45e73fd01b0e4f1dbd40a838cb46d51e (table test-table [id=3bf2e010943645f7a8720a73e77799d1]) successfully deleted
I20260809 20:22:21.869102 27549 catalog_manager.cc:4841] TS b36031d516fa4cb68072050ad948310b (127.5.61.2:43137): tablet 45e73fd01b0e4f1dbd40a838cb46d51e (table test-table [id=3bf2e010943645f7a8720a73e77799d1]) successfully deleted
Aug 09 20:22:21 dist-test-slave-6k22 krb5kdc[27509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.1@KRBTEST.COM
I20260809 20:22:21.880232 28145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:21.840812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51896 (local address 127.5.61.1:38275)
0809 20:22:21.841299 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:21.841306 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:21.856005 (+ 14699us) server_negotiation.cc:407] Connection header received
0809 20:22:21.856466 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:21.856472 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:21.856549 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:21.856636 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:21.858511 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.859979 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.861881 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:21.862643 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:21.862766 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:21.872622 (+ 9856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:21.872639 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:21.872661 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:21.872711 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:21.876873 (+ 4162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:21.877752 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:21.877761 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:21.877764 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:21.877939 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:21.878391 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:21.878396 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:21.878401 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:21.878882 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:21.879191 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:21.879604 (+ 413us) server_negotiation.cc:299] Negotiation successful
0809 20:22:21.879859 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357}
I20260809 20:22:21.880998 27789 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:21.982442 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 27639
I20260809 20:22:22.006364 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 27803
Aug 09 20:22:22 dist-test-slave-6k22 krb5kdc[27509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.2@KRBTEST.COM
I20260809 20:22:22.021046 28148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:22.007189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55864 (local address 127.5.61.2:43137)
0809 20:22:22.007381 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:22.007385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:22.007427 (+ 42us) server_negotiation.cc:407] Connection header received
0809 20:22:22.007487 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:22.007491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:22.007643 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:22.007732 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:22.008924 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:22.009731 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:22.010934 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:22.011333 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:22.011449 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:22.015848 (+ 4399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:22.015862 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:22.015876 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:22.015904 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:22.018510 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:22.019163 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:22.019170 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:22.019173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:22.019289 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:22.019795 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:22.019798 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:22.019800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:22.020209 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:22.020424 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:22.020636 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 20:22:22.020796 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 20:22:22.021790 27960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:22.111166 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 27803
I20260809 20:22:22.133190 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 27974
Aug 09 20:22:22 dist-test-slave-6k22 krb5kdc[27509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.3@KRBTEST.COM
I20260809 20:22:22.147943 28146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:22.133960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49164 (local address 127.5.61.3:35463)
0809 20:22:22.134126 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:22.134130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:22.134149 (+ 19us) server_negotiation.cc:407] Connection header received
0809 20:22:22.134311 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:22.134315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:22.134380 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:22.134466 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:22.135814 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:22.136551 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:22.137813 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:22.138183 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:22.138275 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:22.142759 (+ 4484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:22.142774 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:22.142790 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:22.142827 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:22.145499 (+ 2672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:22.146092 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:22.146099 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:22.146101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:22.146238 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:22.146554 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:22.146557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:22.146559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:22.146901 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:22.147131 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:22.147380 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 20:22:22.147531 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 20:22:22.148577 28121 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:22.240300 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 27974
I20260809 20:22:22.261644 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 27526
I20260809 20:22:22.262813 27603 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:22.372598 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 27526
2026-08-09T20:22:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4628 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T20:22:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:22:22Z Disabled control of system clock
I20260809 20:22:22.463320 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:35501
--webserver_interface=127.5.61.62
--webserver_port=0
--builtin_ntp_servers=127.5.61.20:34705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:35501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 20:22:22.801760 28191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:22.802043 28191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:22.802129 28191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:22.809398 28191 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:22:22.809499 28191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:22.809536 28191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:22.809592 28191 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:22:22.809635 28191 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:22:22.818717 28191 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:34705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:35501
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:35501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:22.819998 28191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:22.821743 28191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:22.836925 28198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:22.837005 28202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:22.837035 28199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:22.838003 28191 server_base.cc:1034] running on GCE node
I20260809 20:22:22.839046 28191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:22.840572 28191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:22.841820 28191 hybrid_clock.cc:648] HybridClock initialized: now 1786306942841770 us; error 60 us; skew 500 ppm
I20260809 20:22:22.842190 28191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:22:22.844447 28191 webserver.cc:460] Webserver started at http://127.5.61.62:41217/ using document root <none> and password file <none>
I20260809 20:22:22.845144 28191 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:22.845281 28191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:22.845600 28191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:22.848562 28191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "20f947aa45214f6097b2a71675273c29"
format_stamp: "Formatted at 2026-08-09 20:22:22 on dist-test-slave-6k22"
server_key: "3a9a967fd4656ac0ad5f35748eef4170"
server_key_iv: "7e8816e36301738c7907263696a8c1a6"
server_key_version: "encryptionkey@0"
I20260809 20:22:22.849395 28191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "20f947aa45214f6097b2a71675273c29"
format_stamp: "Formatted at 2026-08-09 20:22:22 on dist-test-slave-6k22"
server_key: "3a9a967fd4656ac0ad5f35748eef4170"
server_key_iv: "7e8816e36301738c7907263696a8c1a6"
server_key_version: "encryptionkey@0"
I20260809 20:22:22.854554 28191 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 20:22:22.858152 28208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:22.859483 28191 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 20:22:22.859695 28191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "20f947aa45214f6097b2a71675273c29"
format_stamp: "Formatted at 2026-08-09 20:22:22 on dist-test-slave-6k22"
server_key: "3a9a967fd4656ac0ad5f35748eef4170"
server_key_iv: "7e8816e36301738c7907263696a8c1a6"
server_key_version: "encryptionkey@0"
I20260809 20:22:22.859932 28191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:22.880580 28191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:22.881393 28191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:22.881687 28191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:22.901204 28191 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:35501
I20260809 20:22:22.901223 28267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:35501 every 8 connection(s)
I20260809 20:22:22.902683 28191 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:22:22.906320 28268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:22.910279 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 28191
I20260809 20:22:22.910458 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:22:22.911003 5364 external_mini_cluster.cc:1428] Setting key 10b0bc55fe4f40ea87751f5ea4c56b5a
I20260809 20:22:22.917801 28268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20f947aa45214f6097b2a71675273c29: Bootstrap starting.
I20260809 20:22:22.921464 28268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20f947aa45214f6097b2a71675273c29: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:22.922722 28268 log.cc:824] T 00000000000000000000000000000000 P 20f947aa45214f6097b2a71675273c29: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:22.925892 28268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20f947aa45214f6097b2a71675273c29: No bootstrap required, opened a new log
I20260809 20:22:22.925977 28272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:22.912983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53616 (local address 127.5.61.62:35501)
0809 20:22:22.913961 (+ 978us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:22.913972 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:22.914013 (+ 41us) server_negotiation.cc:407] Connection header received
0809 20:22:22.916345 (+ 2332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:22.916377 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:22.916801 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:22.917103 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:22.917757 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:22.919214 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:22.921254 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:22.922001 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:22.922145 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:22.923525 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:22.923539 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:22.923587 (+ 48us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:22.924309 (+ 722us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:22.924387 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:22.924407 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:22.924496 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:22.924732 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 20:22:22.925060 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":401,"threads_started":1}
I20260809 20:22:22.933667 28268 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20f947aa45214f6097b2a71675273c29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20f947aa45214f6097b2a71675273c29" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 35501 } }
I20260809 20:22:22.934055 28268 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20f947aa45214f6097b2a71675273c29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:22.934161 28268 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20f947aa45214f6097b2a71675273c29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f947aa45214f6097b2a71675273c29, State: Initialized, Role: FOLLOWER
I20260809 20:22:22.934671 28268 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20f947aa45214f6097b2a71675273c29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20f947aa45214f6097b2a71675273c29" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 35501 } }
I20260809 20:22:22.934924 28268 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20f947aa45214f6097b2a71675273c29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:22:22.935081 28268 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20f947aa45214f6097b2a71675273c29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:22:22.935251 28268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20f947aa45214f6097b2a71675273c29 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:22.937693 28268 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20f947aa45214f6097b2a71675273c29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20f947aa45214f6097b2a71675273c29" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 35501 } }
I20260809 20:22:22.938268 28268 leader_election.cc:302] T 00000000000000000000000000000000 P 20f947aa45214f6097b2a71675273c29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20f947aa45214f6097b2a71675273c29; no voters:
I20260809 20:22:22.938820 28268 leader_election.cc:288] T 00000000000000000000000000000000 P 20f947aa45214f6097b2a71675273c29 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:22:22.938973 28274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20f947aa45214f6097b2a71675273c29 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:22.939736 28274 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20f947aa45214f6097b2a71675273c29 [term 1 LEADER]: Becoming Leader. State: Replica: 20f947aa45214f6097b2a71675273c29, State: Running, Role: LEADER
I20260809 20:22:22.940294 28274 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20f947aa45214f6097b2a71675273c29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20f947aa45214f6097b2a71675273c29" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 35501 } }
I20260809 20:22:22.940881 28268 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20f947aa45214f6097b2a71675273c29 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:22:22.943308 28276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20f947aa45214f6097b2a71675273c29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20f947aa45214f6097b2a71675273c29. Latest consensus state: current_term: 1 leader_uuid: "20f947aa45214f6097b2a71675273c29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20f947aa45214f6097b2a71675273c29" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 35501 } } }
I20260809 20:22:22.943333 28275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20f947aa45214f6097b2a71675273c29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20f947aa45214f6097b2a71675273c29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20f947aa45214f6097b2a71675273c29" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 35501 } } }
I20260809 20:22:22.943856 28275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20f947aa45214f6097b2a71675273c29 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:22.943862 28276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20f947aa45214f6097b2a71675273c29 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:22.946209 28283 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:22:22.951534 28283 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:22:22.959452 28283 catalog_manager.cc:1347] Generated new cluster ID: 981aac0c47a24379a8c495ee85764ac8
I20260809 20:22:22.959586 28283 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:22:22.978767 28283 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:22:22.980166 28283 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:22:22.991170 28283 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20f947aa45214f6097b2a71675273c29: Generated new TSK 0
I20260809 20:22:22.992030 28283 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:22:22.997095 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.1:0
--local_ip_for_outbound_sockets=127.5.61.1
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:35501
--builtin_ntp_servers=127.5.61.20:34705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 20:22:23.342396 28296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:23.342702 28296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:23.342857 28296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:23.350982 28296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:23.351080 28296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:23.351265 28296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.1
I20260809 20:22:23.361589 28296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:34705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.61.1
--webserver_port=0
--tserver_master_addrs=127.5.61.62:35501
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.1
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:23.362957 28296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:23.365226 28296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:23.379664 28303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:23.379627 28305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:23.379755 28302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:23.380803 28296 server_base.cc:1034] running on GCE node
I20260809 20:22:23.381487 28296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:23.382484 28296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:23.383720 28296 hybrid_clock.cc:648] HybridClock initialized: now 1786306943383621 us; error 49 us; skew 500 ppm
I20260809 20:22:23.384078 28296 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:22:23.386474 28296 webserver.cc:460] Webserver started at http://127.5.61.1:39665/ using document root <none> and password file <none>
I20260809 20:22:23.387230 28296 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:23.387358 28296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:23.387784 28296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:23.390753 28296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-0/data/instance:
uuid: "e499c656ea77465db9b22534ac0daea5"
format_stamp: "Formatted at 2026-08-09 20:22:23 on dist-test-slave-6k22"
server_key: "31e6e2fa3800e754e644d56e3882255d"
server_key_iv: "183b175d2095336f2b8d9e389a702867"
server_key_version: "encryptionkey@0"
I20260809 20:22:23.391659 28296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance:
uuid: "e499c656ea77465db9b22534ac0daea5"
format_stamp: "Formatted at 2026-08-09 20:22:23 on dist-test-slave-6k22"
server_key: "31e6e2fa3800e754e644d56e3882255d"
server_key_iv: "183b175d2095336f2b8d9e389a702867"
server_key_version: "encryptionkey@0"
I20260809 20:22:23.397048 28296 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 20:22:23.400635 28311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:23.401932 28296 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:22:23.402114 28296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-0/wal
uuid: "e499c656ea77465db9b22534ac0daea5"
format_stamp: "Formatted at 2026-08-09 20:22:23 on dist-test-slave-6k22"
server_key: "31e6e2fa3800e754e644d56e3882255d"
server_key_iv: "183b175d2095336f2b8d9e389a702867"
server_key_version: "encryptionkey@0"
I20260809 20:22:23.402348 28296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:23.417277 28296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:23.418047 28296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:23.418334 28296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:23.419291 28296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:22:23.420735 28296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:22:23.420852 28296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:23.420984 28296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:22:23.421068 28296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:23.455097 28296 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.1:37147
I20260809 20:22:23.455197 28448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.1:37147 every 8 connection(s)
I20260809 20:22:23.456471 28296 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-0/data/info.pb
I20260809 20:22:23.459347 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 28296
I20260809 20:22:23.459522 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-0/wal/instance
I20260809 20:22:23.460127 5364 external_mini_cluster.cc:1428] Setting key 1bccc8d0122acd7ecc6eff4412a80f77
I20260809 20:22:23.463116 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.2:0
--local_ip_for_outbound_sockets=127.5.61.2
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:35501
--builtin_ntp_servers=127.5.61.20:34705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 20:22:23.469935 28451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:23.458997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:59149 (local address 127.5.61.62:35501)
0809 20:22:23.459511 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:23.459523 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:23.461015 (+ 1492us) server_negotiation.cc:407] Connection header received
0809 20:22:23.462956 (+ 1941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:23.462961 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:23.463052 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:23.463169 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:23.464376 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:23.465154 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:23.466694 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:23.467122 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:23.467259 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:23.468848 (+ 1589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:23.468856 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:23.468864 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:23.468890 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:23.468968 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:23.468986 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:23.469082 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:23.469383 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 20:22:23.469494 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":250,"threads_started":1}
I20260809 20:22:23.471993 28453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:23.459763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:35501 (local address 127.5.61.1:59149)
0809 20:22:23.460844 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:23.460903 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:23.462683 (+ 1780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:23.463415 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:23.463423 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:23.463510 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:22:23.464148 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:23.464213 (+ 65us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:23.465317 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:23.465324 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:23.466544 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:23.466556 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:23.467351 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:23.467358 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:23.467655 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:23.468467 (+ 812us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 20:22:23.468471 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:22:23.468524 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 20:22:23.468619 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:22:23.469136 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:22:23.469148 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:22:23.469157 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 20:22:23.470142 (+ 985us) client_negotiation.cc:240] Negotiation successful
0809 20:22:23.470563 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":377,"threads_started":1}
I20260809 20:22:23.473590 28449 heartbeater.cc:347] Connected to a master server at 127.5.61.62:35501
I20260809 20:22:23.473925 28449 heartbeater.cc:464] Registering TS with master...
I20260809 20:22:23.474910 28449 heartbeater.cc:511] Master 127.5.61.62:35501 requested a full tablet report, sending...
I20260809 20:22:23.477535 28228 ts_manager.cc:194] Registered new tserver with Master: e499c656ea77465db9b22534ac0daea5 (127.5.61.1:37147)
I20260809 20:22:23.479840 28228 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.61.1:59149
W20260809 20:22:23.810446 28454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:23.810737 28454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:23.810822 28454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:23.817983 28454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:23.818094 28454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:23.818245 28454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.2
I20260809 20:22:23.827207 28454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:34705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.61.2
--webserver_port=0
--tserver_master_addrs=127.5.61.62:35501
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.2
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:23.828392 28454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:23.830080 28454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:23.843371 28461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:23.843416 28464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:23.843626 28460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:23.844278 28454 server_base.cc:1034] running on GCE node
I20260809 20:22:23.844856 28454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:23.845804 28454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:23.847034 28454 hybrid_clock.cc:648] HybridClock initialized: now 1786306943846972 us; error 50 us; skew 500 ppm
I20260809 20:22:23.847385 28454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:22:23.849671 28454 webserver.cc:460] Webserver started at http://127.5.61.2:36101/ using document root <none> and password file <none>
I20260809 20:22:23.850355 28454 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:23.850446 28454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:23.850832 28454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:23.853688 28454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-1/data/instance:
uuid: "000dad0f17a94fae95047f70cfb62fac"
format_stamp: "Formatted at 2026-08-09 20:22:23 on dist-test-slave-6k22"
server_key: "bced5489d286ccef857d6711f780cfba"
server_key_iv: "8f712c0e2efc33676a50d2fa3cf03364"
server_key_version: "encryptionkey@0"
I20260809 20:22:23.854480 28454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance:
uuid: "000dad0f17a94fae95047f70cfb62fac"
format_stamp: "Formatted at 2026-08-09 20:22:23 on dist-test-slave-6k22"
server_key: "bced5489d286ccef857d6711f780cfba"
server_key_iv: "8f712c0e2efc33676a50d2fa3cf03364"
server_key_version: "encryptionkey@0"
I20260809 20:22:23.859450 28454 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260809 20:22:23.862836 28471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:23.864216 28454 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:22:23.864383 28454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-1/wal
uuid: "000dad0f17a94fae95047f70cfb62fac"
format_stamp: "Formatted at 2026-08-09 20:22:23 on dist-test-slave-6k22"
server_key: "bced5489d286ccef857d6711f780cfba"
server_key_iv: "8f712c0e2efc33676a50d2fa3cf03364"
server_key_version: "encryptionkey@0"
I20260809 20:22:23.864610 28454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:23.881824 28454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:23.882628 28454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:23.882936 28454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:23.883903 28454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:22:23.885318 28454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:22:23.885440 28454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:23.885591 28454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:22:23.885689 28454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:23.924649 28454 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.2:43313
I20260809 20:22:23.924702 28621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.2:43313 every 8 connection(s)
I20260809 20:22:23.925935 28454 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-1/data/info.pb
I20260809 20:22:23.926349 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 28454
I20260809 20:22:23.926528 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-1/wal/instance
I20260809 20:22:23.927186 5364 external_mini_cluster.cc:1428] Setting key 96c77ea3f8ace6c5af574d3bddaae590
I20260809 20:22:23.930204 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.61.3:0
--local_ip_for_outbound_sockets=127.5.61.3
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:35501
--builtin_ntp_servers=127.5.61.20:34705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 20:22:23.938414 28451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:23.928551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.2:37791 (local address 127.5.61.62:35501)
0809 20:22:23.928724 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:23.928728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:23.930399 (+ 1671us) server_negotiation.cc:407] Connection header received
0809 20:22:23.932238 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:23.932243 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:23.932337 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:23.932447 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:23.933467 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:23.934176 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:23.935784 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:23.936161 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:23.936300 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:23.937610 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:23.937615 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:23.937618 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:23.937635 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:23.937664 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:23.937676 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:23.937785 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:23.937996 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 20:22:23.938093 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 20:22:23.939328 28626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:23.929192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:35501 (local address 127.5.61.2:37791)
0809 20:22:23.930246 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:23.930306 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:23.931984 (+ 1678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:23.932667 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:23.932675 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:23.932757 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:22:23.933300 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:23.933316 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:23.934354 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:23.934358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:23.935624 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:23.935638 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:23.936385 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:23.936393 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:23.936617 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:23.937303 (+ 686us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 20:22:23.937305 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:22:23.937340 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 20:22:23.937414 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:22:23.937775 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:22:23.937788 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:22:23.937797 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 20:22:23.937981 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 20:22:23.938354 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":376,"threads_started":1}
I20260809 20:22:23.940600 28624 heartbeater.cc:347] Connected to a master server at 127.5.61.62:35501
I20260809 20:22:23.940932 28624 heartbeater.cc:464] Registering TS with master...
I20260809 20:22:23.941785 28624 heartbeater.cc:511] Master 127.5.61.62:35501 requested a full tablet report, sending...
I20260809 20:22:23.943243 28228 ts_manager.cc:194] Registered new tserver with Master: 000dad0f17a94fae95047f70cfb62fac (127.5.61.2:43313)
I20260809 20:22:23.944114 28228 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.61.2:37791
W20260809 20:22:24.271456 28627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:24.271731 28627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:24.271898 28627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:24.278823 28627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:24.278913 28627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:24.279072 28627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.61.3
I20260809 20:22:24.287833 28627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:34705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.61.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.61.3
--webserver_port=0
--tserver_master_addrs=127.5.61.62:35501
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.61.3
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:24.289054 28627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:24.290830 28627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:24.304224 28633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:24.304189 28636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:24.304129 28634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:24.305090 28627 server_base.cc:1034] running on GCE node
I20260809 20:22:24.305802 28627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:24.306833 28627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:24.308048 28627 hybrid_clock.cc:648] HybridClock initialized: now 1786306944308006 us; error 33 us; skew 500 ppm
I20260809 20:22:24.308348 28627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 20:22:24.310616 28627 webserver.cc:460] Webserver started at http://127.5.61.3:42069/ using document root <none> and password file <none>
I20260809 20:22:24.311295 28627 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:24.311448 28627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:24.311812 28627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:24.314586 28627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-2/data/instance:
uuid: "5857a9989a0a40e3b6ab0730dfcaba4f"
format_stamp: "Formatted at 2026-08-09 20:22:24 on dist-test-slave-6k22"
server_key: "dbfb71ca80ccaf966611f4bb6defd931"
server_key_iv: "52263b64b4f056f8b9ae6b48397badb6"
server_key_version: "encryptionkey@0"
I20260809 20:22:24.315383 28627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance:
uuid: "5857a9989a0a40e3b6ab0730dfcaba4f"
format_stamp: "Formatted at 2026-08-09 20:22:24 on dist-test-slave-6k22"
server_key: "dbfb71ca80ccaf966611f4bb6defd931"
server_key_iv: "52263b64b4f056f8b9ae6b48397badb6"
server_key_version: "encryptionkey@0"
I20260809 20:22:24.320503 28627 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 20:22:24.323940 28643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:24.325234 28627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 20:22:24.325407 28627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-2/wal
uuid: "5857a9989a0a40e3b6ab0730dfcaba4f"
format_stamp: "Formatted at 2026-08-09 20:22:24 on dist-test-slave-6k22"
server_key: "dbfb71ca80ccaf966611f4bb6defd931"
server_key_iv: "52263b64b4f056f8b9ae6b48397badb6"
server_key_version: "encryptionkey@0"
I20260809 20:22:24.325624 28627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:24.352093 28627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:24.352936 28627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:24.353227 28627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:24.354223 28627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 20:22:24.355793 28627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 20:22:24.355919 28627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:24.356061 28627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 20:22:24.356148 28627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:24.389398 28627 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.3:36663
I20260809 20:22:24.389488 28776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.3:36663 every 8 connection(s)
I20260809 20:22:24.390726 28627 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-2/data/info.pb
I20260809 20:22:24.391386 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 28627
I20260809 20:22:24.391604 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-2/wal/instance
I20260809 20:22:24.392270 5364 external_mini_cluster.cc:1428] Setting key f1d15be0aae685bc4c3bde9147c5f31b
I20260809 20:22:24.403241 28451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:24.393679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.3:60023 (local address 127.5.61.62:35501)
0809 20:22:24.393861 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:24.393866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:24.395607 (+ 1741us) server_negotiation.cc:407] Connection header received
0809 20:22:24.397365 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:24.397369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:24.397432 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:24.397516 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:24.398558 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.399297 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.400836 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.401284 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.401384 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:24.402577 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:24.402581 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:24.402584 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:24.402601 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:24.402631 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:24.402642 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:24.402713 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:24.402913 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 20:22:24.403014 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 20:22:24.404229 28782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:24.394288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:35501 (local address 127.5.61.3:60023)
0809 20:22:24.395421 (+ 1133us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:24.395476 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:24.397126 (+ 1650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:24.397729 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:24.397749 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:24.397830 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:22:24.398414 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:24.398436 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.399434 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.399437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:24.400691 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:24.400703 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.401464 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.401468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:24.401687 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:24.402300 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 20:22:24.402302 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:22:24.402342 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 20:22:24.402411 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:22:24.402725 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:22:24.402742 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:22:24.402748 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 20:22:24.402883 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 20:22:24.403137 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":349,"threads_started":1}
I20260809 20:22:24.405494 28778 heartbeater.cc:347] Connected to a master server at 127.5.61.62:35501
I20260809 20:22:24.405850 28778 heartbeater.cc:464] Registering TS with master...
I20260809 20:22:24.406610 28778 heartbeater.cc:511] Master 127.5.61.62:35501 requested a full tablet report, sending...
I20260809 20:22:24.408131 28228 ts_manager.cc:194] Registered new tserver with Master: 5857a9989a0a40e3b6ab0730dfcaba4f (127.5.61.3:36663)
I20260809 20:22:24.408900 28228 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.61.3:60023
I20260809 20:22:24.419669 5364 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 20:22:24.429947 28451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:24.423785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53624 (local address 127.5.61.62:35501)
0809 20:22:24.423997 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:24.424006 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:24.424462 (+ 456us) server_negotiation.cc:407] Connection header received
0809 20:22:24.424663 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:24.424668 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:24.424755 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:24.424858 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:24.425373 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.426051 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.427391 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.427904 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.428041 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:24.429084 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:24.429090 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:24.429094 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:24.429117 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:24.429151 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:24.429163 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:24.429290 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:24.429441 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 20:22:24.429528 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 20:22:24.433370 28228 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53624:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 20:22:24.440215 28228 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 20:22:24.461135 28795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:24.452838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:36663 (local address 127.0.0.1:50480)
0809 20:22:24.453323 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:24.453336 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:24.453416 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:24.454598 (+ 1182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:24.454602 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:24.454631 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:24.455368 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:24.455377 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.456874 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.456883 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:24.458443 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:24.458460 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.460162 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.460170 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:24.460389 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:24.460458 (+ 69us) client_negotiation.cc:769] Sending connection context
0809 20:22:24.460578 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 20:22:24.460755 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":25,"thread_start_us":256,"threads_started":1}
I20260809 20:22:24.462236 28796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:24.453166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50480 (local address 127.5.61.3:36663)
0809 20:22:24.453817 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:24.453822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:24.453847 (+ 25us) server_negotiation.cc:407] Connection header received
0809 20:22:24.453913 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:24.453917 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:24.454107 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:24.454315 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:24.455542 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.456731 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.458668 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.459523 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.459716 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:24.460026 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:24.461656 (+ 1630us) server_negotiation.cc:299] Negotiation successful
0809 20:22:24.461830 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":423,"threads_started":1}
I20260809 20:22:24.466800 28701 tablet_service.cc:1467] Processing CreateTablet for tablet f52a423234a84bdb9847ab555824d7d3 (DEFAULT_TABLE table=test-table [id=ef5e5ef2b4ea40d685e0e06f98a04a60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:22:24.468098 28793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:24.451718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37147 (local address 127.0.0.1:50298)
0809 20:22:24.452773 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:24.452814 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:24.452993 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:24.454161 (+ 1168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:24.454168 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:24.454202 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:24.455111 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:24.455132 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.463021 (+ 7889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.463028 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:24.465288 (+ 2260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:24.465303 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.467230 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.467247 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:24.467450 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:24.467486 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 20:22:24.467576 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 20:22:24.467709 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"mutex_wait_us":89,"thread_start_us":339,"threads_started":1}
I20260809 20:22:24.468636 28701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f52a423234a84bdb9847ab555824d7d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:24.469426 28794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:24.451781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50298 (local address 127.5.61.1:37147)
0809 20:22:24.453545 (+ 1764us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:24.453553 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:24.453580 (+ 27us) server_negotiation.cc:407] Connection header received
0809 20:22:24.453679 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:24.453686 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:24.453908 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:24.454143 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:24.459833 (+ 5690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.462863 (+ 3030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.465549 (+ 2686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.467006 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.468497 (+ 1491us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:24.468757 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:24.468840 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 20:22:24.469006 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1610,"thread_start_us":1665,"threads_started":1}
I20260809 20:22:24.469534 28795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:24.454283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:43313 (local address 127.0.0.1:37732)
0809 20:22:24.461367 (+ 7084us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:24.461387 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:24.461485 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:24.462322 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:24.462326 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:24.462353 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:24.462520 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:24.462529 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.464437 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.464443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:24.466403 (+ 1960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:24.466435 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.468643 (+ 2208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.468649 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:24.468868 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:24.468898 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 20:22:24.468991 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 20:22:24.469138 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6998,"mutex_wait_us":1,"thread_start_us":12498,"threads_started":1}
I20260809 20:22:24.470037 28798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:24.454820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37732 (local address 127.5.61.2:43313)
0809 20:22:24.455922 (+ 1102us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:24.455931 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:24.461551 (+ 5620us) server_negotiation.cc:407] Connection header received
0809 20:22:24.461657 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:24.461666 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:24.461879 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:24.462064 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:24.462651 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.464267 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.466611 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.468143 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.468281 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:24.468536 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:24.469141 (+ 605us) server_negotiation.cc:299] Negotiation successful
0809 20:22:24.469364 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":943,"thread_start_us":806,"threads_started":1}
I20260809 20:22:24.472761 28369 tablet_service.cc:1467] Processing CreateTablet for tablet f52a423234a84bdb9847ab555824d7d3 (DEFAULT_TABLE table=test-table [id=ef5e5ef2b4ea40d685e0e06f98a04a60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:22:24.472986 28538 tablet_service.cc:1467] Processing CreateTablet for tablet f52a423234a84bdb9847ab555824d7d3 (DEFAULT_TABLE table=test-table [id=ef5e5ef2b4ea40d685e0e06f98a04a60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 20:22:24.473948 28369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f52a423234a84bdb9847ab555824d7d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:24.474166 28538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f52a423234a84bdb9847ab555824d7d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:24.482899 28449 heartbeater.cc:503] Master 127.5.61.62:35501 was elected leader, sending a full tablet report...
I20260809 20:22:24.488519 28799 tablet_bootstrap.cc:492] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5: Bootstrap starting.
I20260809 20:22:24.490340 28800 tablet_bootstrap.cc:492] T f52a423234a84bdb9847ab555824d7d3 P 000dad0f17a94fae95047f70cfb62fac: Bootstrap starting.
I20260809 20:22:24.493040 28801 tablet_bootstrap.cc:492] T f52a423234a84bdb9847ab555824d7d3 P 5857a9989a0a40e3b6ab0730dfcaba4f: Bootstrap starting.
I20260809 20:22:24.493803 28799 tablet_bootstrap.cc:654] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:24.494076 28800 tablet_bootstrap.cc:654] T f52a423234a84bdb9847ab555824d7d3 P 000dad0f17a94fae95047f70cfb62fac: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:24.495042 28800 log.cc:824] T f52a423234a84bdb9847ab555824d7d3 P 000dad0f17a94fae95047f70cfb62fac: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:24.495138 28799 log.cc:824] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:24.496580 28801 tablet_bootstrap.cc:654] T f52a423234a84bdb9847ab555824d7d3 P 5857a9989a0a40e3b6ab0730dfcaba4f: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:24.497843 28801 log.cc:824] T f52a423234a84bdb9847ab555824d7d3 P 5857a9989a0a40e3b6ab0730dfcaba4f: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:24.498250 28799 tablet_bootstrap.cc:492] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5: No bootstrap required, opened a new log
I20260809 20:22:24.498302 28800 tablet_bootstrap.cc:492] T f52a423234a84bdb9847ab555824d7d3 P 000dad0f17a94fae95047f70cfb62fac: No bootstrap required, opened a new log
I20260809 20:22:24.498584 28799 ts_tablet_manager.cc:1397] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.002s
I20260809 20:22:24.498651 28800 ts_tablet_manager.cc:1397] T f52a423234a84bdb9847ab555824d7d3 P 000dad0f17a94fae95047f70cfb62fac: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260809 20:22:24.501025 28801 tablet_bootstrap.cc:492] T f52a423234a84bdb9847ab555824d7d3 P 5857a9989a0a40e3b6ab0730dfcaba4f: No bootstrap required, opened a new log
I20260809 20:22:24.501257 28801 ts_tablet_manager.cc:1397] T f52a423234a84bdb9847ab555824d7d3 P 5857a9989a0a40e3b6ab0730dfcaba4f: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 20:22:24.506295 28799 raft_consensus.cc:348] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e499c656ea77465db9b22534ac0daea5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37147 } } peers { permanent_uuid: "5857a9989a0a40e3b6ab0730dfcaba4f" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 36663 } } peers { permanent_uuid: "000dad0f17a94fae95047f70cfb62fac" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 43313 } }
I20260809 20:22:24.506654 28799 raft_consensus.cc:374] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:24.506764 28799 raft_consensus.cc:729] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e499c656ea77465db9b22534ac0daea5, State: Initialized, Role: FOLLOWER
I20260809 20:22:24.507393 28799 consensus_queue.cc:260] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e499c656ea77465db9b22534ac0daea5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37147 } } peers { permanent_uuid: "5857a9989a0a40e3b6ab0730dfcaba4f" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 36663 } } peers { permanent_uuid: "000dad0f17a94fae95047f70cfb62fac" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 43313 } }
I20260809 20:22:24.508991 28799 ts_tablet_manager.cc:1428] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5: Time spent starting tablet: real 0.010s user 0.006s sys 0.005s
I20260809 20:22:24.509048 28800 raft_consensus.cc:348] T f52a423234a84bdb9847ab555824d7d3 P 000dad0f17a94fae95047f70cfb62fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e499c656ea77465db9b22534ac0daea5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37147 } } peers { permanent_uuid: "5857a9989a0a40e3b6ab0730dfcaba4f" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 36663 } } peers { permanent_uuid: "000dad0f17a94fae95047f70cfb62fac" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 43313 } }
I20260809 20:22:24.509470 28800 raft_consensus.cc:374] T f52a423234a84bdb9847ab555824d7d3 P 000dad0f17a94fae95047f70cfb62fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:24.509583 28800 raft_consensus.cc:729] T f52a423234a84bdb9847ab555824d7d3 P 000dad0f17a94fae95047f70cfb62fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 000dad0f17a94fae95047f70cfb62fac, State: Initialized, Role: FOLLOWER
I20260809 20:22:24.510178 28800 consensus_queue.cc:260] T f52a423234a84bdb9847ab555824d7d3 P 000dad0f17a94fae95047f70cfb62fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e499c656ea77465db9b22534ac0daea5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37147 } } peers { permanent_uuid: "5857a9989a0a40e3b6ab0730dfcaba4f" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 36663 } } peers { permanent_uuid: "000dad0f17a94fae95047f70cfb62fac" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 43313 } }
I20260809 20:22:24.511264 28624 heartbeater.cc:503] Master 127.5.61.62:35501 was elected leader, sending a full tablet report...
I20260809 20:22:24.511955 28800 ts_tablet_manager.cc:1428] T f52a423234a84bdb9847ab555824d7d3 P 000dad0f17a94fae95047f70cfb62fac: Time spent starting tablet: real 0.013s user 0.009s sys 0.006s
I20260809 20:22:24.512457 28801 raft_consensus.cc:348] T f52a423234a84bdb9847ab555824d7d3 P 5857a9989a0a40e3b6ab0730dfcaba4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e499c656ea77465db9b22534ac0daea5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37147 } } peers { permanent_uuid: "5857a9989a0a40e3b6ab0730dfcaba4f" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 36663 } } peers { permanent_uuid: "000dad0f17a94fae95047f70cfb62fac" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 43313 } }
I20260809 20:22:24.512895 28801 raft_consensus.cc:374] T f52a423234a84bdb9847ab555824d7d3 P 5857a9989a0a40e3b6ab0730dfcaba4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:24.513010 28801 raft_consensus.cc:729] T f52a423234a84bdb9847ab555824d7d3 P 5857a9989a0a40e3b6ab0730dfcaba4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5857a9989a0a40e3b6ab0730dfcaba4f, State: Initialized, Role: FOLLOWER
I20260809 20:22:24.513559 28801 consensus_queue.cc:260] T f52a423234a84bdb9847ab555824d7d3 P 5857a9989a0a40e3b6ab0730dfcaba4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e499c656ea77465db9b22534ac0daea5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37147 } } peers { permanent_uuid: "5857a9989a0a40e3b6ab0730dfcaba4f" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 36663 } } peers { permanent_uuid: "000dad0f17a94fae95047f70cfb62fac" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 43313 } }
I20260809 20:22:24.514916 28778 heartbeater.cc:503] Master 127.5.61.62:35501 was elected leader, sending a full tablet report...
I20260809 20:22:24.515365 28801 ts_tablet_manager.cc:1428] T f52a423234a84bdb9847ab555824d7d3 P 5857a9989a0a40e3b6ab0730dfcaba4f: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
W20260809 20:22:24.642684 28781 tablet.cc:2369] T f52a423234a84bdb9847ab555824d7d3 P 5857a9989a0a40e3b6ab0730dfcaba4f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:22:24.661747 28805 raft_consensus.cc:482] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 20:22:24.662101 28805 raft_consensus.cc:504] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e499c656ea77465db9b22534ac0daea5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37147 } } peers { permanent_uuid: "5857a9989a0a40e3b6ab0730dfcaba4f" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 36663 } } peers { permanent_uuid: "000dad0f17a94fae95047f70cfb62fac" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 43313 } }
I20260809 20:22:24.663942 28805 leader_election.cc:288] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5857a9989a0a40e3b6ab0730dfcaba4f (127.5.61.3:36663), 000dad0f17a94fae95047f70cfb62fac (127.5.61.2:43313)
I20260809 20:22:24.671020 28796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:24.664517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:37243 (local address 127.5.61.3:36663)
0809 20:22:24.664717 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:24.664725 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:24.665477 (+ 752us) server_negotiation.cc:407] Connection header received
0809 20:22:24.665676 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:24.665681 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:24.665754 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:24.665851 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:24.666503 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.667209 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.669318 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.670127 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.670280 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:24.670395 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:24.670643 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 20:22:24.670779 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 20:22:24.671132 28798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:24.664463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.61.1:60837 (local address 127.5.61.2:43313)
0809 20:22:24.664686 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:24.664695 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:24.665383 (+ 688us) server_negotiation.cc:407] Connection header received
0809 20:22:24.665619 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:24.665624 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:24.665683 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:24.665791 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 20:22:24.666441 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.667132 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.669483 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.670229 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.670379 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:24.670471 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:24.670761 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 20:22:24.670883 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 20:22:24.671219 28810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:24.664403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.3:36663 (local address 127.5.61.1:37243)
0809 20:22:24.665363 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:24.665378 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:24.665532 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:24.665885 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:24.665890 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:24.665916 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:24.666322 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:24.666332 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.667350 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.667355 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:24.669123 (+ 1768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:24.669135 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.670316 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.670322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:24.670482 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:24.670506 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 20:22:24.670616 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 20:22:24.670751 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"mutex_wait_us":60,"thread_start_us":580,"threads_started":1}
I20260809 20:22:24.671284 28809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:24.664381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.2:43313 (local address 127.5.61.1:60837)
0809 20:22:24.665260 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:24.665279 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 20:22:24.665458 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:22:24.665806 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:22:24.665810 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:22:24.665833 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 20:22:24.666289 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:24.666299 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.667273 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.667278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:24.669264 (+ 1986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:22:24.669276 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.670422 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.670428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:22:24.670583 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:24.670605 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 20:22:24.670707 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 20:22:24.670833 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"mutex_wait_us":97,"thread_start_us":432,"threads_started":1}
I20260809 20:22:24.672533 28724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f52a423234a84bdb9847ab555824d7d3" candidate_uuid: "e499c656ea77465db9b22534ac0daea5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5857a9989a0a40e3b6ab0730dfcaba4f" is_pre_election: true
I20260809 20:22:24.672557 28564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f52a423234a84bdb9847ab555824d7d3" candidate_uuid: "e499c656ea77465db9b22534ac0daea5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "000dad0f17a94fae95047f70cfb62fac" is_pre_election: true
I20260809 20:22:24.673034 28564 raft_consensus.cc:2454] T f52a423234a84bdb9847ab555824d7d3 P 000dad0f17a94fae95047f70cfb62fac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e499c656ea77465db9b22534ac0daea5 in term 0.
I20260809 20:22:24.673039 28724 raft_consensus.cc:2454] T f52a423234a84bdb9847ab555824d7d3 P 5857a9989a0a40e3b6ab0730dfcaba4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e499c656ea77465db9b22534ac0daea5 in term 0.
I20260809 20:22:24.673707 28313 leader_election.cc:302] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 000dad0f17a94fae95047f70cfb62fac, 5857a9989a0a40e3b6ab0730dfcaba4f, e499c656ea77465db9b22534ac0daea5; no voters:
I20260809 20:22:24.674111 28805 raft_consensus.cc:2781] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 20:22:24.674257 28805 raft_consensus.cc:482] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 20:22:24.674358 28805 raft_consensus.cc:3037] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:24.676642 28805 raft_consensus.cc:504] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e499c656ea77465db9b22534ac0daea5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37147 } } peers { permanent_uuid: "5857a9989a0a40e3b6ab0730dfcaba4f" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 36663 } } peers { permanent_uuid: "000dad0f17a94fae95047f70cfb62fac" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 43313 } }
I20260809 20:22:24.677302 28805 leader_election.cc:288] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5 [CANDIDATE]: Term 1 election: Requested vote from peers 5857a9989a0a40e3b6ab0730dfcaba4f (127.5.61.3:36663), 000dad0f17a94fae95047f70cfb62fac (127.5.61.2:43313)
I20260809 20:22:24.677672 28724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f52a423234a84bdb9847ab555824d7d3" candidate_uuid: "e499c656ea77465db9b22534ac0daea5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5857a9989a0a40e3b6ab0730dfcaba4f"
I20260809 20:22:24.677842 28564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f52a423234a84bdb9847ab555824d7d3" candidate_uuid: "e499c656ea77465db9b22534ac0daea5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "000dad0f17a94fae95047f70cfb62fac"
I20260809 20:22:24.677911 28724 raft_consensus.cc:3037] T f52a423234a84bdb9847ab555824d7d3 P 5857a9989a0a40e3b6ab0730dfcaba4f [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:24.678048 28564 raft_consensus.cc:3037] T f52a423234a84bdb9847ab555824d7d3 P 000dad0f17a94fae95047f70cfb62fac [term 0 FOLLOWER]: Advancing to term 1
W20260809 20:22:24.679847 28625 tablet.cc:2369] T f52a423234a84bdb9847ab555824d7d3 P 000dad0f17a94fae95047f70cfb62fac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:22:24.681202 28724 raft_consensus.cc:2454] T f52a423234a84bdb9847ab555824d7d3 P 5857a9989a0a40e3b6ab0730dfcaba4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e499c656ea77465db9b22534ac0daea5 in term 1.
I20260809 20:22:24.681376 28564 raft_consensus.cc:2454] T f52a423234a84bdb9847ab555824d7d3 P 000dad0f17a94fae95047f70cfb62fac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e499c656ea77465db9b22534ac0daea5 in term 1.
I20260809 20:22:24.681741 28315 leader_election.cc:302] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5857a9989a0a40e3b6ab0730dfcaba4f, e499c656ea77465db9b22534ac0daea5; no voters:
I20260809 20:22:24.682051 28805 raft_consensus.cc:2781] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:24.682452 28805 raft_consensus.cc:686] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5 [term 1 LEADER]: Becoming Leader. State: Replica: e499c656ea77465db9b22534ac0daea5, State: Running, Role: LEADER
I20260809 20:22:24.682879 28805 consensus_queue.cc:237] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e499c656ea77465db9b22534ac0daea5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37147 } } peers { permanent_uuid: "5857a9989a0a40e3b6ab0730dfcaba4f" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 36663 } } peers { permanent_uuid: "000dad0f17a94fae95047f70cfb62fac" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 43313 } }
I20260809 20:22:24.687131 28227 catalog_manager.cc:5495] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e499c656ea77465db9b22534ac0daea5 (127.5.61.1). New cstate: current_term: 1 leader_uuid: "e499c656ea77465db9b22534ac0daea5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e499c656ea77465db9b22534ac0daea5" member_type: VOTER last_known_addr { host: "127.5.61.1" port: 37147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5857a9989a0a40e3b6ab0730dfcaba4f" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 36663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "000dad0f17a94fae95047f70cfb62fac" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 43313 } health_report { overall_health: UNKNOWN } } }
W20260809 20:22:24.708865 28450 tablet.cc:2369] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:22:24.709755 28794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:24.704310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50314 (local address 127.5.61.1:37147)
0809 20:22:24.704483 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:24.704488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:24.704627 (+ 139us) server_negotiation.cc:407] Connection header received
0809 20:22:24.704799 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:24.704803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:24.704901 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:24.705067 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:22:24.705375 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.706088 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.707491 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.707964 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.708058 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:24.708501 (+ 443us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:22:24.709186 (+ 685us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:22:24.709366 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:24.709441 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 20:22:24.709532 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 20:22:24.717437 28564 raft_consensus.cc:1261] T f52a423234a84bdb9847ab555824d7d3 P 000dad0f17a94fae95047f70cfb62fac [term 1 FOLLOWER]: Refusing update from remote peer e499c656ea77465db9b22534ac0daea5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:22:24.717463 28724 raft_consensus.cc:1261] T f52a423234a84bdb9847ab555824d7d3 P 5857a9989a0a40e3b6ab0730dfcaba4f [term 1 FOLLOWER]: Refusing update from remote peer e499c656ea77465db9b22534ac0daea5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:22:24.718397 28811 consensus_queue.cc:1035] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5857a9989a0a40e3b6ab0730dfcaba4f" member_type: VOTER last_known_addr { host: "127.5.61.3" port: 36663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:22:24.718721 28805 consensus_queue.cc:1035] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "000dad0f17a94fae95047f70cfb62fac" member_type: VOTER last_known_addr { host: "127.5.61.2" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:22:24.743005 28228 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:22:24.743271 28228 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 20:22:24.746745 28228 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 20f947aa45214f6097b2a71675273c29: Sending DeleteTablet for 3 replicas of tablet f52a423234a84bdb9847ab555824d7d3
I20260809 20:22:24.748114 28538 tablet_service.cc:1514] Processing DeleteTablet for tablet f52a423234a84bdb9847ab555824d7d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:22:24 UTC) from {username='slave'} at 127.0.0.1:37732
I20260809 20:22:24.748250 28701 tablet_service.cc:1514] Processing DeleteTablet for tablet f52a423234a84bdb9847ab555824d7d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:22:24 UTC) from {username='slave'} at 127.0.0.1:50480
I20260809 20:22:24.748260 28369 tablet_service.cc:1514] Processing DeleteTablet for tablet f52a423234a84bdb9847ab555824d7d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 20:22:24 UTC) from {username='slave'} at 127.0.0.1:50298
I20260809 20:22:24.749974 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 28296
I20260809 20:22:24.776762 28828 tablet_replica.cc:330] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5: stopping tablet replica
I20260809 20:22:24.780628 28828 raft_consensus.cc:2229] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 20:22:24.781455 28828 raft_consensus.cc:2258] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:22:24.784827 28828 ts_tablet_manager.cc:1905] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:22:24.786113 28826 tablet_replica.cc:330] T f52a423234a84bdb9847ab555824d7d3 P 000dad0f17a94fae95047f70cfb62fac: stopping tablet replica
I20260809 20:22:24.786540 28826 raft_consensus.cc:2229] T f52a423234a84bdb9847ab555824d7d3 P 000dad0f17a94fae95047f70cfb62fac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:22:24.787065 28826 raft_consensus.cc:2258] T f52a423234a84bdb9847ab555824d7d3 P 000dad0f17a94fae95047f70cfb62fac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:22:24.790807 28826 ts_tablet_manager.cc:1905] T f52a423234a84bdb9847ab555824d7d3 P 000dad0f17a94fae95047f70cfb62fac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:22:24.795257 28828 ts_tablet_manager.cc:1918] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:22:24.795426 28828 log.cc:1196] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-0/wal/wals/f52a423234a84bdb9847ab555824d7d3
I20260809 20:22:24.796335 28794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:24.751332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50316 (local address 127.5.61.1:37147)
0809 20:22:24.751524 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:24.751531 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:24.774883 (+ 23352us) server_negotiation.cc:407] Connection header received
0809 20:22:24.775365 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:24.775371 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:24.775476 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:24.775588 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:24.777044 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.778281 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.781944 (+ 3663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.782615 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.782723 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:24.787909 (+ 5186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:24.787923 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:24.787944 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:24.788911 (+ 967us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:24.788995 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:24.789027 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:24.789168 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:24.795858 (+ 6690us) server_negotiation.cc:299] Negotiation successful
0809 20:22:24.795959 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 20:22:24.797493 28443 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:24.797861 28827 tablet_replica.cc:330] T f52a423234a84bdb9847ab555824d7d3 P 5857a9989a0a40e3b6ab0730dfcaba4f: stopping tablet replica
I20260809 20:22:24.798322 28827 raft_consensus.cc:2229] T f52a423234a84bdb9847ab555824d7d3 P 5857a9989a0a40e3b6ab0730dfcaba4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 20:22:24.799000 28827 raft_consensus.cc:2258] T f52a423234a84bdb9847ab555824d7d3 P 5857a9989a0a40e3b6ab0730dfcaba4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 20:22:24.801102 28826 ts_tablet_manager.cc:1918] T f52a423234a84bdb9847ab555824d7d3 P 000dad0f17a94fae95047f70cfb62fac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:22:24.801373 28826 log.cc:1196] T f52a423234a84bdb9847ab555824d7d3 P 000dad0f17a94fae95047f70cfb62fac: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-1/wal/wals/f52a423234a84bdb9847ab555824d7d3
I20260809 20:22:24.802080 28827 ts_tablet_manager.cc:1905] T f52a423234a84bdb9847ab555824d7d3 P 5857a9989a0a40e3b6ab0730dfcaba4f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 20:22:24.802315 28826 ts_tablet_manager.cc:1939] T f52a423234a84bdb9847ab555824d7d3 P 000dad0f17a94fae95047f70cfb62fac: Deleting consensus metadata
I20260809 20:22:24.804294 28209 catalog_manager.cc:4841] TS 000dad0f17a94fae95047f70cfb62fac (127.5.61.2:43313): tablet f52a423234a84bdb9847ab555824d7d3 (table test-table [id=ef5e5ef2b4ea40d685e0e06f98a04a60]) successfully deleted
I20260809 20:22:24.807061 28827 ts_tablet_manager.cc:1918] T f52a423234a84bdb9847ab555824d7d3 P 5857a9989a0a40e3b6ab0730dfcaba4f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 20:22:24.807175 28827 log.cc:1196] T f52a423234a84bdb9847ab555824d7d3 P 5857a9989a0a40e3b6ab0730dfcaba4f: Deleting WAL directory at /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786306623011809-5364-0/minicluster-data/ts-2/wal/wals/f52a423234a84bdb9847ab555824d7d3
I20260809 20:22:24.807838 28827 ts_tablet_manager.cc:1939] T f52a423234a84bdb9847ab555824d7d3 P 5857a9989a0a40e3b6ab0730dfcaba4f: Deleting consensus metadata
I20260809 20:22:24.809463 28212 catalog_manager.cc:4841] TS 5857a9989a0a40e3b6ab0730dfcaba4f (127.5.61.3:36663): tablet f52a423234a84bdb9847ab555824d7d3 (table test-table [id=ef5e5ef2b4ea40d685e0e06f98a04a60]) successfully deleted
I20260809 20:22:24.897746 28828 ts_tablet_manager.cc:1939] T f52a423234a84bdb9847ab555824d7d3 P e499c656ea77465db9b22534ac0daea5: Deleting consensus metadata
I20260809 20:22:24.897862 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 28296
I20260809 20:22:24.921581 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 28454
W20260809 20:22:24.921480 28212 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.61.1:37147 (error 108)
I20260809 20:22:24.924139 28795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:24.923481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37147 (local address 127.0.0.1:50326)
0809 20:22:24.923796 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:24.923913 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.1:37147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
W20260809 20:22:24.924376 28212 catalog_manager.cc:4584] TS e499c656ea77465db9b22534ac0daea5 (127.5.61.1:37147): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f52a423234a84bdb9847ab555824d7d3: Network error: Client connection negotiation failed: client connection to 127.5.61.1:37147: connect: Connection refused (error 111)
I20260809 20:22:24.929673 28798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:24.922478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37742 (local address 127.5.61.2:43313)
0809 20:22:24.922682 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:24.922689 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:24.922711 (+ 22us) server_negotiation.cc:407] Connection header received
0809 20:22:24.922866 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:24.922871 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:24.922983 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:24.923098 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:24.923652 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.924803 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.926574 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:24.927005 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:24.927094 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:24.928192 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:24.928200 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:24.928212 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:24.928843 (+ 631us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:24.928888 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:24.928911 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:24.928996 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:24.929197 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 20:22:24.929310 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 20:22:24.930406 28616 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:24.977975 28795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:24.977478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37147 (local address 127.0.0.1:50332)
0809 20:22:24.977711 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:24.977797 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.1:37147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260809 20:22:24.979001 28795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:24.978661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37147 (local address 127.0.0.1:50346)
0809 20:22:24.978808 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:24.978899 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.1:37147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 20:22:25.018850 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 28454
I20260809 20:22:25.039463 28795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:25.038996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37147 (local address 127.0.0.1:50354)
0809 20:22:25.039215 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:25.039318 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.1:37147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260809 20:22:25.040112 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 28627
I20260809 20:22:25.040900 28795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:25.040479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37147 (local address 127.0.0.1:50366)
0809 20:22:25.040641 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:25.040719 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.1:37147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 20:22:25.047338 28796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:25.040881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50496 (local address 127.5.61.3:36663)
0809 20:22:25.041071 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:25.041079 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:25.041109 (+ 30us) server_negotiation.cc:407] Connection header received
0809 20:22:25.041327 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:25.041339 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:25.041563 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:25.041668 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:25.042105 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:25.042801 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:25.044269 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:25.044683 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:25.044821 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:25.045857 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:25.045870 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:25.045891 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:25.046586 (+ 695us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:25.046634 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:25.046660 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:25.046794 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:25.046930 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 20:22:25.047036 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 20:22:25.048024 28772 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:25.121778 28795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:25.121277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37147 (local address 127.0.0.1:50368)
0809 20:22:25.121524 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:25.121607 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.1:37147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260809 20:22:25.122944 28795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:25.122504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.1:37147 (local address 127.0.0.1:50372)
0809 20:22:25.122713 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 20:22:25.122787 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.1:37147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260809 20:22:25.134526 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 28627
I20260809 20:22:25.156284 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 28191
I20260809 20:22:25.157325 28263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:25.262975 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 28191
2026-08-09T20:22:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2888 ms)
[----------] 6 tests from EncryptionPolicyTest (22551 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:22:25 dist-test-slave-6k22 krb5kdc[28841](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:22:25 dist-test-slave-6k22 krb5kdc[28841](info): set up 2 sockets
Aug 09 20:22:25 dist-test-slave-6k22 krb5kdc[28841](info): commencing operation
krb5kdc: starting...
W20260809 20:22:26.459460 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.004s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:22:26 dist-test-slave-6k22 krb5kdc[28841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:22:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:22:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 20:22:26.758898 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36359
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:27.088967 28856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:27.089293 28856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:27.089470 28856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:27.096576 28856 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:22:27.096717 28856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:27.096817 28856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:27.096939 28856 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:22:27.097018 28856 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:22:27.106629 28856 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36359
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:27.108044 28856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:27.109387 28856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:27.121484 28864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:27.121454 28865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:27.121531 28867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:27.122295 28856 server_base.cc:1034] running on GCE node
I20260809 20:22:27.122983 28856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:27.124346 28856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:27.125543 28856 hybrid_clock.cc:648] HybridClock initialized: now 1786306947125503 us; error 37 us; skew 500 ppm
I20260809 20:22:27.125886 28856 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:27 dist-test-slave-6k22 krb5kdc[28841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306947, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:27.129027 28856 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 20:22:27.130517 28856 webserver.cc:460] Webserver started at http://127.0.0.1:32955/ using document root <none> and password file <none>
I20260809 20:22:27.131213 28856 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:27.131371 28856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:27.131742 28856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:27.134634 28856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "b1ed2e74893c430697e40cc8b212abd3"
format_stamp: "Formatted at 2026-08-09 20:22:27 on dist-test-slave-6k22"
server_key: "b3a7281649ce269fe4d1a5263bec57a7"
server_key_iv: "b0767b2b4ccd54f225dcd6f81ff2ff6d"
server_key_version: "encryptionkey@0"
I20260809 20:22:27.135418 28856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "b1ed2e74893c430697e40cc8b212abd3"
format_stamp: "Formatted at 2026-08-09 20:22:27 on dist-test-slave-6k22"
server_key: "b3a7281649ce269fe4d1a5263bec57a7"
server_key_iv: "b0767b2b4ccd54f225dcd6f81ff2ff6d"
server_key_version: "encryptionkey@0"
I20260809 20:22:27.140369 28856 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 20:22:27.143883 28875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:27.145116 28856 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 20:22:27.145305 28856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "b1ed2e74893c430697e40cc8b212abd3"
format_stamp: "Formatted at 2026-08-09 20:22:27 on dist-test-slave-6k22"
server_key: "b3a7281649ce269fe4d1a5263bec57a7"
server_key_iv: "b0767b2b4ccd54f225dcd6f81ff2ff6d"
server_key_version: "encryptionkey@0"
I20260809 20:22:27.145524 28856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:27.174678 28856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:27.178769 28856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:27.179062 28856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:27.198292 28856 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36359
I20260809 20:22:27.198390 28944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36359 every 8 connection(s)
I20260809 20:22:27.199621 28856 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:22:27.203079 28945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:27.205017 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 28856
I20260809 20:22:27.205252 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:22:27.206074 5364 external_mini_cluster.cc:1428] Setting key 998d023c63e40cb5cefb8f0c11c67d8d
I20260809 20:22:27.215338 28945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1ed2e74893c430697e40cc8b212abd3: Bootstrap starting.
I20260809 20:22:27.219066 28945 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1ed2e74893c430697e40cc8b212abd3: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:27.220316 28945 log.cc:824] T 00000000000000000000000000000000 P b1ed2e74893c430697e40cc8b212abd3: Log is configured to *not* fsync() on all Append() calls
Aug 09 20:22:27 dist-test-slave-6k22 krb5kdc[28841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 20:22:27.223111 28945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1ed2e74893c430697e40cc8b212abd3: No bootstrap required, opened a new log
I20260809 20:22:27.229487 28948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:27.207988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47527 (local address 127.0.0.1:36359)
0809 20:22:27.208839 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:27.208849 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:27.208891 (+ 42us) server_negotiation.cc:407] Connection header received
0809 20:22:27.210497 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:27.210517 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:27.210871 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:27.211172 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:27.212934 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:27.214131 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:27.215979 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:27.216596 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:27.216734 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:27.221951 (+ 5217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:27.221958 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:27.221970 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:27.222007 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:27.225314 (+ 3307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:27.226152 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:27.226161 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:27.226166 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:27.226335 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:27.226818 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:27.226822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:27.226827 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:27.227327 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:27.227719 (+ 392us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:27.228225 (+ 506us) server_negotiation.cc:299] Negotiation successful
0809 20:22:27.228530 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":404,"threads_started":1}
I20260809 20:22:27.230685 28945 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1ed2e74893c430697e40cc8b212abd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1ed2e74893c430697e40cc8b212abd3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36359 } }
I20260809 20:22:27.231019 28945 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1ed2e74893c430697e40cc8b212abd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:27.231145 28945 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1ed2e74893c430697e40cc8b212abd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1ed2e74893c430697e40cc8b212abd3, State: Initialized, Role: FOLLOWER
I20260809 20:22:27.231778 28945 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1ed2e74893c430697e40cc8b212abd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1ed2e74893c430697e40cc8b212abd3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36359 } }
I20260809 20:22:27.232102 28945 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1ed2e74893c430697e40cc8b212abd3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:22:27.232259 28945 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1ed2e74893c430697e40cc8b212abd3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:22:27.232437 28945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1ed2e74893c430697e40cc8b212abd3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:27.233558 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:22:27.234788 28945 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1ed2e74893c430697e40cc8b212abd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1ed2e74893c430697e40cc8b212abd3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36359 } }
I20260809 20:22:27.235256 28945 leader_election.cc:302] T 00000000000000000000000000000000 P b1ed2e74893c430697e40cc8b212abd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1ed2e74893c430697e40cc8b212abd3; no voters:
I20260809 20:22:27.235970 28945 leader_election.cc:288] T 00000000000000000000000000000000 P b1ed2e74893c430697e40cc8b212abd3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:22:27.236122 28950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1ed2e74893c430697e40cc8b212abd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:27.237784 28945 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1ed2e74893c430697e40cc8b212abd3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:22:27.242112 28950 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1ed2e74893c430697e40cc8b212abd3 [term 1 LEADER]: Becoming Leader. State: Replica: b1ed2e74893c430697e40cc8b212abd3, State: Running, Role: LEADER
I20260809 20:22:27.242866 28950 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1ed2e74893c430697e40cc8b212abd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1ed2e74893c430697e40cc8b212abd3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36359 } }
I20260809 20:22:27.246376 28951 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1ed2e74893c430697e40cc8b212abd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1ed2e74893c430697e40cc8b212abd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1ed2e74893c430697e40cc8b212abd3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36359 } } }
I20260809 20:22:27.246892 28951 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1ed2e74893c430697e40cc8b212abd3 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:27.247514 28955 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1ed2e74893c430697e40cc8b212abd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1ed2e74893c430697e40cc8b212abd3. Latest consensus state: current_term: 1 leader_uuid: "b1ed2e74893c430697e40cc8b212abd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1ed2e74893c430697e40cc8b212abd3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36359 } } }
I20260809 20:22:27.247884 28955 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1ed2e74893c430697e40cc8b212abd3 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:27.269375 28974 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:22:27.278292 28948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:27.246179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53585 (local address 127.0.0.1:36359)
0809 20:22:27.246357 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:27.246363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:27.258385 (+ 12022us) server_negotiation.cc:407] Connection header received
0809 20:22:27.258942 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:27.258950 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:27.259047 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:27.259129 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:27.261793 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:27.263242 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:27.265748 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:27.266658 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:27.266774 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:27.270502 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:27.270509 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:27.270513 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:27.270548 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:27.274496 (+ 3948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:27.275497 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:27.275503 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:27.275506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:27.275715 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:27.276318 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:27.276325 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:27.276330 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:27.276726 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:27.277013 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:27.277714 (+ 701us) server_negotiation.cc:299] Negotiation successful
0809 20:22:27.277921 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 20:22:27.283986 28974 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:22:27.292639 28974 catalog_manager.cc:1347] Generated new cluster ID: 23ba6daa2cdd481e98a3281319839842
I20260809 20:22:27.292758 28974 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:22:27.302264 28974 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:22:27.303450 28974 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:22:27.312064 28974 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1ed2e74893c430697e40cc8b212abd3: Generated new TSK 0
I20260809 20:22:27.312831 28974 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:22:27.320075 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 28856
I20260809 20:22:27.321622 28939 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:27.411347 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 28856
W20260809 20:22:27.431931 28839 connection.cc:535] client connection to 127.0.0.1:36359 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36359): Connection reset by peer (error 104)
2026-08-09T20:22:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2143 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:22:27 dist-test-slave-6k22 krb5kdc[28984](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:22:27 dist-test-slave-6k22 krb5kdc[28984](info): set up 2 sockets
Aug 09 20:22:27 dist-test-slave-6k22 krb5kdc[28984](info): commencing operation
krb5kdc: starting...
W20260809 20:22:28.595088 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.005s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:22:28 dist-test-slave-6k22 krb5kdc[28984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:22:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:22:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 20:22:28.887197 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32999
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:29.216297 29001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:29.216578 29001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:29.216786 29001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:29.223865 29001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:22:29.223999 29001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:29.224081 29001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:29.224192 29001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:22:29.224284 29001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:22:29.233800 29001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32999
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:29.235083 29001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:29.236380 29001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:29.248884 29009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:29.248939 29008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:29.248996 29013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:29.249836 29001 server_base.cc:1034] running on GCE node
I20260809 20:22:29.250510 29001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:29.251827 29001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:29.253024 29001 hybrid_clock.cc:648] HybridClock initialized: now 1786306949252988 us; error 37 us; skew 500 ppm
I20260809 20:22:29.253355 29001 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:29 dist-test-slave-6k22 krb5kdc[28984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306949, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:29.257270 29001 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 20:22:29.258772 29001 webserver.cc:460] Webserver started at http://127.0.0.1:46849/ using document root <none> and password file <none>
I20260809 20:22:29.259449 29001 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:29.259640 29001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:29.259954 29001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:29.262749 29001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "59e48eb05bd547b7b7dd711a515fb2c7"
format_stamp: "Formatted at 2026-08-09 20:22:29 on dist-test-slave-6k22"
server_key: "e11065cf474d509fecfae30ce6b64fb7"
server_key_iv: "e4788215609c2fb9555f3e767503f998"
server_key_version: "encryptionkey@0"
I20260809 20:22:29.263521 29001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "59e48eb05bd547b7b7dd711a515fb2c7"
format_stamp: "Formatted at 2026-08-09 20:22:29 on dist-test-slave-6k22"
server_key: "e11065cf474d509fecfae30ce6b64fb7"
server_key_iv: "e4788215609c2fb9555f3e767503f998"
server_key_version: "encryptionkey@0"
I20260809 20:22:29.268672 29001 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 20:22:29.272351 29020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:29.273833 29001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:22:29.274019 29001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "59e48eb05bd547b7b7dd711a515fb2c7"
format_stamp: "Formatted at 2026-08-09 20:22:29 on dist-test-slave-6k22"
server_key: "e11065cf474d509fecfae30ce6b64fb7"
server_key_iv: "e4788215609c2fb9555f3e767503f998"
server_key_version: "encryptionkey@0"
I20260809 20:22:29.274209 29001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:29.317354 29001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:29.321381 29001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:29.321681 29001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:29.340216 29001 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32999
I20260809 20:22:29.340266 29086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32999 every 8 connection(s)
I20260809 20:22:29.341574 29001 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:22:29.345252 29087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:29.346138 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 29001
I20260809 20:22:29.346326 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:22:29.346791 5364 external_mini_cluster.cc:1428] Setting key cb3a4fe56d677ab5c6d0c926cc9c659d
I20260809 20:22:29.359789 29087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59e48eb05bd547b7b7dd711a515fb2c7: Bootstrap starting.
I20260809 20:22:29.361781 29090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:29.348692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44031 (local address 127.0.0.1:32999)
0809 20:22:29.349468 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:29.349482 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:29.349535 (+ 53us) server_negotiation.cc:407] Connection header received
0809 20:22:29.351872 (+ 2337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:29.351899 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:29.352404 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:29.352835 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:29.353324 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:29.355176 (+ 1852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:29.356535 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:29.357623 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:29.357775 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:29.359018 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:29.359028 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:29.359046 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:29.359802 (+ 756us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:29.359924 (+ 122us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:29.359967 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:29.360147 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:29.360383 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 20:22:29.360550 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":298,"threads_started":1}
I20260809 20:22:29.363772 29087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59e48eb05bd547b7b7dd711a515fb2c7: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:29.365120 29087 log.cc:824] T 00000000000000000000000000000000 P 59e48eb05bd547b7b7dd711a515fb2c7: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:29.365845 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:22:29.368232 29087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59e48eb05bd547b7b7dd711a515fb2c7: No bootstrap required, opened a new log
I20260809 20:22:29.377697 29090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:29.371201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57471 (local address 127.0.0.1:32999)
0809 20:22:29.371384 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:29.371389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:29.371724 (+ 335us) server_negotiation.cc:407] Connection header received
0809 20:22:29.371889 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:29.371895 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:29.371984 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:29.372085 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:29.372534 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:29.373641 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:29.374962 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:29.375701 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:29.375829 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:29.376921 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:29.376925 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:29.376929 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:29.376945 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:29.377000 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:29.377014 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:29.377149 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:29.377356 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 20:22:29.377412 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 20:22:29.378362 29087 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59e48eb05bd547b7b7dd711a515fb2c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59e48eb05bd547b7b7dd711a515fb2c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32999 } }
I20260809 20:22:29.378925 29087 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59e48eb05bd547b7b7dd711a515fb2c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:29.379040 29087 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59e48eb05bd547b7b7dd711a515fb2c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59e48eb05bd547b7b7dd711a515fb2c7, State: Initialized, Role: FOLLOWER
I20260809 20:22:29.379743 29087 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59e48eb05bd547b7b7dd711a515fb2c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59e48eb05bd547b7b7dd711a515fb2c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32999 } }
I20260809 20:22:29.379976 29087 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59e48eb05bd547b7b7dd711a515fb2c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:22:29.380074 29087 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59e48eb05bd547b7b7dd711a515fb2c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:22:29.380193 29087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59e48eb05bd547b7b7dd711a515fb2c7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:29.382555 29087 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59e48eb05bd547b7b7dd711a515fb2c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59e48eb05bd547b7b7dd711a515fb2c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32999 } }
I20260809 20:22:29.383175 29087 leader_election.cc:302] T 00000000000000000000000000000000 P 59e48eb05bd547b7b7dd711a515fb2c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59e48eb05bd547b7b7dd711a515fb2c7; no voters:
I20260809 20:22:29.383750 29087 leader_election.cc:288] T 00000000000000000000000000000000 P 59e48eb05bd547b7b7dd711a515fb2c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:22:29.383910 29098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59e48eb05bd547b7b7dd711a515fb2c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:29.384692 29098 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59e48eb05bd547b7b7dd711a515fb2c7 [term 1 LEADER]: Becoming Leader. State: Replica: 59e48eb05bd547b7b7dd711a515fb2c7, State: Running, Role: LEADER
I20260809 20:22:29.385262 29098 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59e48eb05bd547b7b7dd711a515fb2c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59e48eb05bd547b7b7dd711a515fb2c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32999 } }
I20260809 20:22:29.385998 29087 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59e48eb05bd547b7b7dd711a515fb2c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:22:29.388211 29100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59e48eb05bd547b7b7dd711a515fb2c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59e48eb05bd547b7b7dd711a515fb2c7. Latest consensus state: current_term: 1 leader_uuid: "59e48eb05bd547b7b7dd711a515fb2c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59e48eb05bd547b7b7dd711a515fb2c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32999 } } }
I20260809 20:22:29.388533 29100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59e48eb05bd547b7b7dd711a515fb2c7 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:29.390794 29103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:22:29.390617 29099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59e48eb05bd547b7b7dd711a515fb2c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59e48eb05bd547b7b7dd711a515fb2c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59e48eb05bd547b7b7dd711a515fb2c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32999 } } }
I20260809 20:22:29.390944 29099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59e48eb05bd547b7b7dd711a515fb2c7 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:29.396427 29103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:22:29.406100 29103 catalog_manager.cc:1347] Generated new cluster ID: 383959eb480f436aa997c484560d9b38
I20260809 20:22:29.406213 29103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:22:29.418076 29103 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:22:29.419344 29103 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:22:29.426254 29103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59e48eb05bd547b7b7dd711a515fb2c7: Generated new TSK 0
I20260809 20:22:29.427068 29103 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:22:29.440375 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 29001
W20260809 20:22:29.441352 29082 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44031
W20260809 20:22:29.442133 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 29001: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 20:22:29.442332 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 29001
W20260809 20:22:29.459553 28982 connection.cc:535] client connection to 127.0.0.1:32999 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32999): Connection reset by peer (error 104)
2026-08-09T20:22:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2027 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:22:29 dist-test-slave-6k22 krb5kdc[29123](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:22:29 dist-test-slave-6k22 krb5kdc[29123](info): set up 2 sockets
Aug 09 20:22:29 dist-test-slave-6k22 krb5kdc[29123](info): commencing operation
krb5kdc: starting...
W20260809 20:22:30.682922 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:22:30 dist-test-slave-6k22 krb5kdc[29123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:22:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:22:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 20:22:30.990851 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36197
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:31.319491 29139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:31.319801 29139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:31.319972 29139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:31.326912 29139 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:22:31.327008 29139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:31.327092 29139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:31.327173 29139 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:22:31.327282 29139 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:22:31.336493 29139 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36197
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:31.337759 29139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:31.339062 29139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:31.350517 29144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:31.350618 29145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:31.350591 29147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:31.351877 29139 server_base.cc:1034] running on GCE node
I20260809 20:22:31.352626 29139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:31.353842 29139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:31.355082 29139 hybrid_clock.cc:648] HybridClock initialized: now 1786306951355041 us; error 47 us; skew 500 ppm
I20260809 20:22:31.355423 29139 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:31 dist-test-slave-6k22 krb5kdc[29123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306951, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:31.358680 29139 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 20:22:31.360064 29139 webserver.cc:460] Webserver started at http://127.0.0.1:38295/ using document root <none> and password file <none>
I20260809 20:22:31.360708 29139 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:31.360805 29139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:31.361040 29139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:31.363848 29139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "85cf68147c854f5ba1617d75f04a6792"
format_stamp: "Formatted at 2026-08-09 20:22:31 on dist-test-slave-6k22"
server_key: "c1b3e3d90d8a82eb31ef616ecfd22c0c"
server_key_iv: "7f206fd9f85b212f6b093e4e58471220"
server_key_version: "encryptionkey@0"
I20260809 20:22:31.364581 29139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "85cf68147c854f5ba1617d75f04a6792"
format_stamp: "Formatted at 2026-08-09 20:22:31 on dist-test-slave-6k22"
server_key: "c1b3e3d90d8a82eb31ef616ecfd22c0c"
server_key_iv: "7f206fd9f85b212f6b093e4e58471220"
server_key_version: "encryptionkey@0"
I20260809 20:22:31.369540 29139 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 20:22:31.372942 29155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:31.374190 29139 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 20:22:31.374323 29139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "85cf68147c854f5ba1617d75f04a6792"
format_stamp: "Formatted at 2026-08-09 20:22:31 on dist-test-slave-6k22"
server_key: "c1b3e3d90d8a82eb31ef616ecfd22c0c"
server_key_iv: "7f206fd9f85b212f6b093e4e58471220"
server_key_version: "encryptionkey@0"
I20260809 20:22:31.374500 29139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:31.380128 29139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:31.384285 29139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:31.384539 29139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:31.402484 29139 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36197
I20260809 20:22:31.402534 29218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36197 every 8 connection(s)
I20260809 20:22:31.403887 29139 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:22:31.407428 29219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:31.408743 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 29139
I20260809 20:22:31.408921 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:22:31.409364 5364 external_mini_cluster.cc:1428] Setting key eb99c9f327a0a8c11bc54b44e5f80626
I20260809 20:22:31.416810 29223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:31.411122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49921 (local address 127.0.0.1:36197)
0809 20:22:31.411926 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:31.411936 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:31.411973 (+ 37us) server_negotiation.cc:407] Connection header received
0809 20:22:31.413551 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:31.413572 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:31.413916 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:31.414291 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:31.415316 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:31.415322 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:31.415334 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:31.415349 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:31.415404 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:31.415427 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:31.415694 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:31.415822 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 20:22:31.415990 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":390,"threads_started":1}
I20260809 20:22:31.420397 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:22:31.420722 29219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85cf68147c854f5ba1617d75f04a6792: Bootstrap starting.
I20260809 20:22:31.423784 29219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85cf68147c854f5ba1617d75f04a6792: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:31.424942 29219 log.cc:824] T 00000000000000000000000000000000 P 85cf68147c854f5ba1617d75f04a6792: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:31.427443 29223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:31.424666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49555 (local address 127.0.0.1:36197)
0809 20:22:31.424870 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:31.424878 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:31.425391 (+ 513us) server_negotiation.cc:407] Connection header received
0809 20:22:31.425653 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:31.425659 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:31.425739 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:31.425875 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:31.426795 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:31.426800 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:31.426804 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:31.426822 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:31.426863 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:31.426886 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:31.427002 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:31.427096 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 20:22:31.427177 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 20:22:31.428247 29219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85cf68147c854f5ba1617d75f04a6792: No bootstrap required, opened a new log
I20260809 20:22:31.435482 29219 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85cf68147c854f5ba1617d75f04a6792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85cf68147c854f5ba1617d75f04a6792" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36197 } }
I20260809 20:22:31.435910 29219 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85cf68147c854f5ba1617d75f04a6792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:31.435985 29219 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85cf68147c854f5ba1617d75f04a6792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85cf68147c854f5ba1617d75f04a6792, State: Initialized, Role: FOLLOWER
I20260809 20:22:31.436466 29219 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85cf68147c854f5ba1617d75f04a6792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85cf68147c854f5ba1617d75f04a6792" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36197 } }
I20260809 20:22:31.436719 29219 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85cf68147c854f5ba1617d75f04a6792 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:22:31.436826 29219 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85cf68147c854f5ba1617d75f04a6792 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:22:31.436949 29219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85cf68147c854f5ba1617d75f04a6792 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:31.439025 29219 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85cf68147c854f5ba1617d75f04a6792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85cf68147c854f5ba1617d75f04a6792" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36197 } }
I20260809 20:22:31.439579 29219 leader_election.cc:302] T 00000000000000000000000000000000 P 85cf68147c854f5ba1617d75f04a6792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85cf68147c854f5ba1617d75f04a6792; no voters:
I20260809 20:22:31.440289 29219 leader_election.cc:288] T 00000000000000000000000000000000 P 85cf68147c854f5ba1617d75f04a6792 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:22:31.440459 29230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85cf68147c854f5ba1617d75f04a6792 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:31.441118 29230 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85cf68147c854f5ba1617d75f04a6792 [term 1 LEADER]: Becoming Leader. State: Replica: 85cf68147c854f5ba1617d75f04a6792, State: Running, Role: LEADER
I20260809 20:22:31.441632 29230 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85cf68147c854f5ba1617d75f04a6792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85cf68147c854f5ba1617d75f04a6792" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36197 } }
I20260809 20:22:31.442274 29219 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85cf68147c854f5ba1617d75f04a6792 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:22:31.444585 29232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85cf68147c854f5ba1617d75f04a6792 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85cf68147c854f5ba1617d75f04a6792" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85cf68147c854f5ba1617d75f04a6792" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36197 } } }
I20260809 20:22:31.445434 29232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85cf68147c854f5ba1617d75f04a6792 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:31.444880 29233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85cf68147c854f5ba1617d75f04a6792 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85cf68147c854f5ba1617d75f04a6792. Latest consensus state: current_term: 1 leader_uuid: "85cf68147c854f5ba1617d75f04a6792" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85cf68147c854f5ba1617d75f04a6792" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36197 } } }
I20260809 20:22:31.445927 29233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85cf68147c854f5ba1617d75f04a6792 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:31.447151 29242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:22:31.452355 29242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:22:31.461246 29242 catalog_manager.cc:1347] Generated new cluster ID: e4ad8d0eb7644e5aa8a6318a9b31863e
I20260809 20:22:31.461380 29242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:22:31.474601 29242 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:22:31.474995 29242 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:22:31.481839 29242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85cf68147c854f5ba1617d75f04a6792: Generated new TSK 0
I20260809 20:22:31.482560 29242 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:22:31.487468 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 29139
W20260809 20:22:31.489077 29213 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49921
W20260809 20:22:31.489856 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 29139: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 20:22:31.490016 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 29139
2026-08-09T20:22:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2050 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:22:31 dist-test-slave-6k22 krb5kdc[29255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:22:31 dist-test-slave-6k22 krb5kdc[29255](info): set up 2 sockets
Aug 09 20:22:31 dist-test-slave-6k22 krb5kdc[29255](info): commencing operation
krb5kdc: starting...
W20260809 20:22:32.661413 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:22:32 dist-test-slave-6k22 krb5kdc[29255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:22:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:22:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 20:22:32.954334 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46287
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46287
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:33.286217 29269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:33.286540 29269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:33.286645 29269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:33.293610 29269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:22:33.293721 29269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:33.293767 29269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:33.293820 29269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:22:33.293861 29269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:22:33.303467 29269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:33.304728 29269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:33.305961 29269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:33.318128 29275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:33.318100 29276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:33.318208 29279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:33.323964 29269 server_base.cc:1034] running on GCE node
I20260809 20:22:33.324734 29269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:33.326083 29269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:33.327316 29269 hybrid_clock.cc:648] HybridClock initialized: now 1786306953327270 us; error 47 us; skew 500 ppm
I20260809 20:22:33.327711 29269 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:33 dist-test-slave-6k22 krb5kdc[29255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306953, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:33.330687 29269 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 20:22:33.332278 29269 webserver.cc:460] Webserver started at http://127.0.0.1:37353/ using document root <none> and password file <none>
I20260809 20:22:33.332995 29269 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:33.333145 29269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:33.333469 29269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:33.336429 29269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "6bcb3f8d232c483e91880d6e18392c2f"
format_stamp: "Formatted at 2026-08-09 20:22:33 on dist-test-slave-6k22"
server_key: "6155b8c84748b9f92b6e12e2bdd1c7a6"
server_key_iv: "c8ec8fd42cd269eb13fe1a9c1b8d59df"
server_key_version: "encryptionkey@0"
I20260809 20:22:33.337162 29269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "6bcb3f8d232c483e91880d6e18392c2f"
format_stamp: "Formatted at 2026-08-09 20:22:33 on dist-test-slave-6k22"
server_key: "6155b8c84748b9f92b6e12e2bdd1c7a6"
server_key_iv: "c8ec8fd42cd269eb13fe1a9c1b8d59df"
server_key_version: "encryptionkey@0"
I20260809 20:22:33.342417 29269 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 20:22:33.346071 29287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:33.347283 29269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 20:22:33.347481 29269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "6bcb3f8d232c483e91880d6e18392c2f"
format_stamp: "Formatted at 2026-08-09 20:22:33 on dist-test-slave-6k22"
server_key: "6155b8c84748b9f92b6e12e2bdd1c7a6"
server_key_iv: "c8ec8fd42cd269eb13fe1a9c1b8d59df"
server_key_version: "encryptionkey@0"
I20260809 20:22:33.347728 29269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:33.364908 29269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:33.369329 29269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:33.369655 29269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:33.388943 29269 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46287
I20260809 20:22:33.388976 29346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46287 every 8 connection(s)
I20260809 20:22:33.390334 29269 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:22:33.393802 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 29269
I20260809 20:22:33.394003 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:22:33.394255 29347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:33.394562 5364 external_mini_cluster.cc:1428] Setting key 4b7f92e26d6293d3014438c897fbed8c
I20260809 20:22:33.408691 29347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bcb3f8d232c483e91880d6e18392c2f: Bootstrap starting.
Aug 09 20:22:33 dist-test-slave-6k22 krb5kdc[29255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 20:22:33.412497 29347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bcb3f8d232c483e91880d6e18392c2f: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:33.413913 29347 log.cc:824] T 00000000000000000000000000000000 P 6bcb3f8d232c483e91880d6e18392c2f: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:33.417512 29347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bcb3f8d232c483e91880d6e18392c2f: No bootstrap required, opened a new log
I20260809 20:22:33.419335 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:33.396408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48949 (local address 127.0.0.1:46287)
0809 20:22:33.397259 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:33.397276 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:33.397332 (+ 56us) server_negotiation.cc:407] Connection header received
0809 20:22:33.399301 (+ 1969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:33.399329 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:33.399917 (+ 588us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:33.400355 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:33.401705 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:33.403661 (+ 1956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:33.404939 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:33.406062 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:33.406256 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:33.410381 (+ 4125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:33.410392 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:33.410411 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:33.410465 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:33.414503 (+ 4038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:33.415492 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:33.415503 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:33.415514 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:33.415729 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:33.416133 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:33.416137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:33.416140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:33.416875 (+ 735us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:33.417174 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:33.417527 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 20:22:33.417971 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":347,"threads_started":1}
I20260809 20:22:33.423260 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:22:33.426462 29347 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bcb3f8d232c483e91880d6e18392c2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bcb3f8d232c483e91880d6e18392c2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46287 } }
I20260809 20:22:33.426837 29347 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6bcb3f8d232c483e91880d6e18392c2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:33.426957 29347 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bcb3f8d232c483e91880d6e18392c2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bcb3f8d232c483e91880d6e18392c2f, State: Initialized, Role: FOLLOWER
I20260809 20:22:33.427795 29347 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bcb3f8d232c483e91880d6e18392c2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bcb3f8d232c483e91880d6e18392c2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46287 } }
I20260809 20:22:33.428107 29347 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6bcb3f8d232c483e91880d6e18392c2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:22:33.428268 29347 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bcb3f8d232c483e91880d6e18392c2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:22:33.428565 29347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bcb3f8d232c483e91880d6e18392c2f [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:33.431802 29347 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bcb3f8d232c483e91880d6e18392c2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bcb3f8d232c483e91880d6e18392c2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46287 } }
I20260809 20:22:33.432431 29347 leader_election.cc:302] T 00000000000000000000000000000000 P 6bcb3f8d232c483e91880d6e18392c2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6bcb3f8d232c483e91880d6e18392c2f; no voters:
I20260809 20:22:33.433117 29347 leader_election.cc:288] T 00000000000000000000000000000000 P 6bcb3f8d232c483e91880d6e18392c2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:22:33.433223 29359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bcb3f8d232c483e91880d6e18392c2f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:33.435393 29347 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bcb3f8d232c483e91880d6e18392c2f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:22:33.435545 29359 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bcb3f8d232c483e91880d6e18392c2f [term 1 LEADER]: Becoming Leader. State: Replica: 6bcb3f8d232c483e91880d6e18392c2f, State: Running, Role: LEADER
I20260809 20:22:33.436113 29359 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bcb3f8d232c483e91880d6e18392c2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bcb3f8d232c483e91880d6e18392c2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46287 } }
I20260809 20:22:33.438006 29362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bcb3f8d232c483e91880d6e18392c2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bcb3f8d232c483e91880d6e18392c2f. Latest consensus state: current_term: 1 leader_uuid: "6bcb3f8d232c483e91880d6e18392c2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bcb3f8d232c483e91880d6e18392c2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46287 } } }
I20260809 20:22:33.438409 29362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bcb3f8d232c483e91880d6e18392c2f [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:33.438908 29360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bcb3f8d232c483e91880d6e18392c2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6bcb3f8d232c483e91880d6e18392c2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bcb3f8d232c483e91880d6e18392c2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46287 } } }
I20260809 20:22:33.439280 29360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bcb3f8d232c483e91880d6e18392c2f [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:33.441985 29367 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:22:33.448498 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:33.428502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49673 (local address 127.0.0.1:46287)
0809 20:22:33.428664 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:33.428671 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:33.428716 (+ 45us) server_negotiation.cc:407] Connection header received
0809 20:22:33.428952 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:33.428955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:33.429015 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:33.429089 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:33.430990 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:33.431723 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:33.433770 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:33.434429 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:33.434550 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:33.438360 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:33.438382 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:33.438394 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:33.438447 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:33.443513 (+ 5066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:33.444640 (+ 1127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:33.444649 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:33.444655 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:33.444838 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:33.445381 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:33.445388 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:33.445394 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:33.445854 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:33.447549 (+ 1695us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:33.447763 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 20:22:33.448075 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 20:22:33.457492 29367 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:22:33.466037 29367 catalog_manager.cc:1347] Generated new cluster ID: cc2f3e0f45264e41b9a0ab0abf84b7b8
I20260809 20:22:33.466150 29367 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:22:33.485409 29367 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:22:33.486680 29367 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:22:33.491777 29367 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6bcb3f8d232c483e91880d6e18392c2f: Generated new TSK 0
I20260809 20:22:33.492574 29367 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:22:33.499045 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 29269
I20260809 20:22:33.500674 29341 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:33.589641 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 29269
2026-08-09T20:22:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2098 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:22:33 dist-test-slave-6k22 krb5kdc[29387](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:22:33 dist-test-slave-6k22 krb5kdc[29387](info): set up 2 sockets
Aug 09 20:22:33 dist-test-slave-6k22 krb5kdc[29387](info): commencing operation
krb5kdc: starting...
W20260809 20:22:34.783836 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:22:34 dist-test-slave-6k22 krb5kdc[29387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:22:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:22:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 20:22:35.083904 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40115
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:35.413311 29402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:35.413651 29402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:35.413820 29402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:35.421269 29402 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:22:35.421443 29402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:35.421558 29402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:35.421689 29402 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:22:35.421777 29402 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:22:35.432260 29402 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:35.433696 29402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:35.435125 29402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:35.447220 29408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:35.447234 29409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:35.447153 29412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:35.448390 29402 server_base.cc:1034] running on GCE node
I20260809 20:22:35.449160 29402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:35.450650 29402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:35.451891 29402 hybrid_clock.cc:648] HybridClock initialized: now 1786306955451835 us; error 48 us; skew 500 ppm
I20260809 20:22:35.452263 29402 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:35 dist-test-slave-6k22 krb5kdc[29387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306955, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:35.455725 29402 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 20:22:35.457259 29402 webserver.cc:460] Webserver started at http://127.0.0.1:39369/ using document root <none> and password file <none>
I20260809 20:22:35.457934 29402 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:35.458045 29402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:35.458320 29402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:35.461397 29402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "1b0cbcc8d9ba4c469cb6ecca4a847b4c"
format_stamp: "Formatted at 2026-08-09 20:22:35 on dist-test-slave-6k22"
server_key: "82c41573990ac488fca6ec854872bbec"
server_key_iv: "e1761ca65ad12e610c2ba7800790452e"
server_key_version: "encryptionkey@0"
I20260809 20:22:35.462126 29402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "1b0cbcc8d9ba4c469cb6ecca4a847b4c"
format_stamp: "Formatted at 2026-08-09 20:22:35 on dist-test-slave-6k22"
server_key: "82c41573990ac488fca6ec854872bbec"
server_key_iv: "e1761ca65ad12e610c2ba7800790452e"
server_key_version: "encryptionkey@0"
I20260809 20:22:35.467439 29402 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 20:22:35.471168 29420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:35.472604 29402 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:22:35.472841 29402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "1b0cbcc8d9ba4c469cb6ecca4a847b4c"
format_stamp: "Formatted at 2026-08-09 20:22:35 on dist-test-slave-6k22"
server_key: "82c41573990ac488fca6ec854872bbec"
server_key_iv: "e1761ca65ad12e610c2ba7800790452e"
server_key_version: "encryptionkey@0"
I20260809 20:22:35.473047 29402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:35.486078 29402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:35.491120 29402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:35.491461 29402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:35.511678 29402 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40115
I20260809 20:22:35.511771 29482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40115 every 8 connection(s)
I20260809 20:22:35.513274 29402 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:22:35.517055 29483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:35.522776 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 29402
I20260809 20:22:35.523000 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:22:35.523594 5364 external_mini_cluster.cc:1428] Setting key a8ee3f59b320eea2d68cc6af625891c6
I20260809 20:22:35.528296 29483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0cbcc8d9ba4c469cb6ecca4a847b4c: Bootstrap starting.
I20260809 20:22:35.531886 29483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b0cbcc8d9ba4c469cb6ecca4a847b4c: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:35.533138 29483 log.cc:824] T 00000000000000000000000000000000 P 1b0cbcc8d9ba4c469cb6ecca4a847b4c: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:35.536453 29483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0cbcc8d9ba4c469cb6ecca4a847b4c: No bootstrap required, opened a new log
I20260809 20:22:35.537338 29487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:35.525456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43415 (local address 127.0.0.1:40115)
0809 20:22:35.526372 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:35.526383 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:35.526423 (+ 40us) server_negotiation.cc:407] Connection header received
0809 20:22:35.528107 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:35.528147 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:35.528596 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:35.528910 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:35.529505 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:35.530769 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:35.532918 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:35.533775 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:35.534001 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:35.535052 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:35.535059 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:35.535072 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:35.535651 (+ 579us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:35.535722 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:35.535750 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:35.535837 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:35.536077 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 20:22:35.536349 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":459,"threads_started":1}
I20260809 20:22:35.541270 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:22:35.546607 29483 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b0cbcc8d9ba4c469cb6ecca4a847b4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0cbcc8d9ba4c469cb6ecca4a847b4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40115 } }
I20260809 20:22:35.547053 29483 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b0cbcc8d9ba4c469cb6ecca4a847b4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:35.547170 29483 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b0cbcc8d9ba4c469cb6ecca4a847b4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b0cbcc8d9ba4c469cb6ecca4a847b4c, State: Initialized, Role: FOLLOWER
I20260809 20:22:35.547914 29483 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b0cbcc8d9ba4c469cb6ecca4a847b4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0cbcc8d9ba4c469cb6ecca4a847b4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40115 } }
I20260809 20:22:35.548231 29483 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b0cbcc8d9ba4c469cb6ecca4a847b4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:22:35.548404 29483 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b0cbcc8d9ba4c469cb6ecca4a847b4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:22:35.548614 29483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b0cbcc8d9ba4c469cb6ecca4a847b4c [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:35.551775 29483 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b0cbcc8d9ba4c469cb6ecca4a847b4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0cbcc8d9ba4c469cb6ecca4a847b4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40115 } }
I20260809 20:22:35.552377 29483 leader_election.cc:302] T 00000000000000000000000000000000 P 1b0cbcc8d9ba4c469cb6ecca4a847b4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b0cbcc8d9ba4c469cb6ecca4a847b4c; no voters:
I20260809 20:22:35.553063 29483 leader_election.cc:288] T 00000000000000000000000000000000 P 1b0cbcc8d9ba4c469cb6ecca4a847b4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:22:35.553196 29495 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b0cbcc8d9ba4c469cb6ecca4a847b4c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:35.553824 29487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:35.546589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53401 (local address 127.0.0.1:40115)
0809 20:22:35.546771 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:35.546776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:35.547181 (+ 405us) server_negotiation.cc:407] Connection header received
0809 20:22:35.547407 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:35.547411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:35.547492 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:35.547640 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:35.548249 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:35.549169 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:35.551166 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:35.551683 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:35.551779 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:35.552921 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:35.552929 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:35.552935 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:35.552959 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:35.553017 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:35.553039 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:35.553157 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:35.553332 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 20:22:35.553429 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 20:22:35.554594 29495 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b0cbcc8d9ba4c469cb6ecca4a847b4c [term 1 LEADER]: Becoming Leader. State: Replica: 1b0cbcc8d9ba4c469cb6ecca4a847b4c, State: Running, Role: LEADER
I20260809 20:22:35.555299 29495 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b0cbcc8d9ba4c469cb6ecca4a847b4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0cbcc8d9ba4c469cb6ecca4a847b4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40115 } }
I20260809 20:22:35.555639 29483 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b0cbcc8d9ba4c469cb6ecca4a847b4c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:22:35.558297 29497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b0cbcc8d9ba4c469cb6ecca4a847b4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b0cbcc8d9ba4c469cb6ecca4a847b4c. Latest consensus state: current_term: 1 leader_uuid: "1b0cbcc8d9ba4c469cb6ecca4a847b4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0cbcc8d9ba4c469cb6ecca4a847b4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40115 } } }
I20260809 20:22:35.558918 29497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b0cbcc8d9ba4c469cb6ecca4a847b4c [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:35.558596 29496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b0cbcc8d9ba4c469cb6ecca4a847b4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b0cbcc8d9ba4c469cb6ecca4a847b4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0cbcc8d9ba4c469cb6ecca4a847b4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40115 } } }
I20260809 20:22:35.559455 29496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b0cbcc8d9ba4c469cb6ecca4a847b4c [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:35.560459 29506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:22:35.565359 29506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:22:35.574760 29506 catalog_manager.cc:1347] Generated new cluster ID: e09b3c71f6ec4298a11deaca13996af5
I20260809 20:22:35.574935 29506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:22:35.593004 29506 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:22:35.594319 29506 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:22:35.603948 29506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b0cbcc8d9ba4c469cb6ecca4a847b4c: Generated new TSK 0
I20260809 20:22:35.604813 29506 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:22:35.610954 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 29402
W20260809 20:22:35.612023 29478 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43415
W20260809 20:22:35.612699 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 29402: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 20:22:35.612848 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 29402
2026-08-09T20:22:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2024 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:22:35 dist-test-slave-6k22 krb5kdc[29522](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:22:35 dist-test-slave-6k22 krb5kdc[29522](info): set up 2 sockets
Aug 09 20:22:35 dist-test-slave-6k22 krb5kdc[29522](info): commencing operation
krb5kdc: starting...
W20260809 20:22:36.803423 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:22:36 dist-test-slave-6k22 krb5kdc[29522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:22:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:22:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 20:22:37.098577 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40405
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:37.430764 29538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:37.431067 29538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:37.431268 29538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:37.438634 29538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:22:37.438768 29538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:37.438877 29538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:37.438990 29538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:22:37.439077 29538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:22:37.448719 29538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:37.449962 29538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:37.451270 29538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:37.463649 29544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:37.463543 29543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:37.463610 29546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:37.464582 29538 server_base.cc:1034] running on GCE node
I20260809 20:22:37.465372 29538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:37.466753 29538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:37.467957 29538 hybrid_clock.cc:648] HybridClock initialized: now 1786306957467913 us; error 41 us; skew 500 ppm
I20260809 20:22:37.468252 29538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:37 dist-test-slave-6k22 krb5kdc[29522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306957, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:37.477852 29538 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 20:22:37.479413 29538 webserver.cc:460] Webserver started at http://127.0.0.1:38195/ using document root <none> and password file <none>
I20260809 20:22:37.480209 29538 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:37.480350 29538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:37.480687 29538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:37.483824 29538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "93624d0d921548ea92e8c323b57382c0"
format_stamp: "Formatted at 2026-08-09 20:22:37 on dist-test-slave-6k22"
server_key: "acc87f9d8c25d566cdda85b13cffd71c"
server_key_iv: "c5ad997f1cfabe9c34a79e76d4fdc841"
server_key_version: "encryptionkey@0"
I20260809 20:22:37.484622 29538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "93624d0d921548ea92e8c323b57382c0"
format_stamp: "Formatted at 2026-08-09 20:22:37 on dist-test-slave-6k22"
server_key: "acc87f9d8c25d566cdda85b13cffd71c"
server_key_iv: "c5ad997f1cfabe9c34a79e76d4fdc841"
server_key_version: "encryptionkey@0"
I20260809 20:22:37.489998 29538 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 20:22:37.493585 29556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:37.494915 29538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 20:22:37.495158 29538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "93624d0d921548ea92e8c323b57382c0"
format_stamp: "Formatted at 2026-08-09 20:22:37 on dist-test-slave-6k22"
server_key: "acc87f9d8c25d566cdda85b13cffd71c"
server_key_iv: "c5ad997f1cfabe9c34a79e76d4fdc841"
server_key_version: "encryptionkey@0"
I20260809 20:22:37.495370 29538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:37.501242 29538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:37.505209 29538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:37.505504 29538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:37.524320 29538 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40405
I20260809 20:22:37.524411 29619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40405 every 8 connection(s)
I20260809 20:22:37.525789 29538 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:22:37.528270 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 29538
I20260809 20:22:37.528463 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:22:37.529021 5364 external_mini_cluster.cc:1428] Setting key 86e255b7a60fff4ce7f0af9b16d5fd36
I20260809 20:22:37.530140 29620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:37.536550 29623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:37.530813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42873 (local address 127.0.0.1:40405)
0809 20:22:37.531689 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:37.531700 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:37.531740 (+ 40us) server_negotiation.cc:407] Connection header received
0809 20:22:37.533230 (+ 1490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:37.533250 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:37.533588 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:37.533922 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:37.534972 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:37.534979 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:37.534991 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:37.535015 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:37.535082 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:37.535105 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:37.535265 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:37.535401 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 20:22:37.535638 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":356,"threads_started":1}
I20260809 20:22:37.540251 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:22:37.543052 29620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93624d0d921548ea92e8c323b57382c0: Bootstrap starting.
I20260809 20:22:37.546738 29620 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93624d0d921548ea92e8c323b57382c0: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:37.547119 29623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:37.544705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44363 (local address 127.0.0.1:40405)
0809 20:22:37.544905 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:37.544910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:37.545206 (+ 296us) server_negotiation.cc:407] Connection header received
0809 20:22:37.545357 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:37.545362 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:37.545407 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:37.545482 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:37.546436 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:37.546440 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:37.546443 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:37.546457 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:37.546492 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:37.546506 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:37.546590 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:37.546746 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 20:22:37.546789 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 20:22:37.547976 29620 log.cc:824] T 00000000000000000000000000000000 P 93624d0d921548ea92e8c323b57382c0: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:37.550544 29620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93624d0d921548ea92e8c323b57382c0: No bootstrap required, opened a new log
I20260809 20:22:37.557495 29620 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93624d0d921548ea92e8c323b57382c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93624d0d921548ea92e8c323b57382c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40405 } }
I20260809 20:22:37.557837 29620 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93624d0d921548ea92e8c323b57382c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:37.557945 29620 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93624d0d921548ea92e8c323b57382c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93624d0d921548ea92e8c323b57382c0, State: Initialized, Role: FOLLOWER
I20260809 20:22:37.558565 29620 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93624d0d921548ea92e8c323b57382c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93624d0d921548ea92e8c323b57382c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40405 } }
I20260809 20:22:37.558796 29620 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93624d0d921548ea92e8c323b57382c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:22:37.558938 29620 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93624d0d921548ea92e8c323b57382c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:22:37.559106 29620 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93624d0d921548ea92e8c323b57382c0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:37.561270 29620 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93624d0d921548ea92e8c323b57382c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93624d0d921548ea92e8c323b57382c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40405 } }
I20260809 20:22:37.561772 29620 leader_election.cc:302] T 00000000000000000000000000000000 P 93624d0d921548ea92e8c323b57382c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93624d0d921548ea92e8c323b57382c0; no voters:
I20260809 20:22:37.562333 29620 leader_election.cc:288] T 00000000000000000000000000000000 P 93624d0d921548ea92e8c323b57382c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:22:37.562489 29632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93624d0d921548ea92e8c323b57382c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:37.563186 29632 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93624d0d921548ea92e8c323b57382c0 [term 1 LEADER]: Becoming Leader. State: Replica: 93624d0d921548ea92e8c323b57382c0, State: Running, Role: LEADER
I20260809 20:22:37.563876 29632 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93624d0d921548ea92e8c323b57382c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93624d0d921548ea92e8c323b57382c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40405 } }
I20260809 20:22:37.564512 29620 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93624d0d921548ea92e8c323b57382c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:22:37.567014 29633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93624d0d921548ea92e8c323b57382c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93624d0d921548ea92e8c323b57382c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93624d0d921548ea92e8c323b57382c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40405 } } }
I20260809 20:22:37.567672 29633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93624d0d921548ea92e8c323b57382c0 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:37.567056 29634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93624d0d921548ea92e8c323b57382c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93624d0d921548ea92e8c323b57382c0. Latest consensus state: current_term: 1 leader_uuid: "93624d0d921548ea92e8c323b57382c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93624d0d921548ea92e8c323b57382c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40405 } } }
I20260809 20:22:37.568142 29634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93624d0d921548ea92e8c323b57382c0 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:37.569310 29640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:22:37.574559 29640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:22:37.583621 29640 catalog_manager.cc:1347] Generated new cluster ID: 22fe79afd8234cd0b082f58c83c588f0
I20260809 20:22:37.583729 29640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:22:37.599335 29640 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:22:37.599814 29640 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:22:37.610322 29640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93624d0d921548ea92e8c323b57382c0: Generated new TSK 0
I20260809 20:22:37.611239 29640 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:22:37.617328 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 29538
W20260809 20:22:37.618341 29614 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42873
W20260809 20:22:37.619160 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 29538: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 20:22:37.619365 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 29538
2026-08-09T20:22:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2003 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:22:37 dist-test-slave-6k22 krb5kdc[29654](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:22:37 dist-test-slave-6k22 krb5kdc[29654](info): set up 2 sockets
Aug 09 20:22:37 dist-test-slave-6k22 krb5kdc[29654](info): commencing operation
krb5kdc: starting...
W20260809 20:22:38.820799 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.003s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:22:39 dist-test-slave-6k22 krb5kdc[29654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:22:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:22:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 20:22:39.126695 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38639
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:39.460505 29669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:39.460776 29669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:39.460865 29669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:39.467967 29669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:22:39.468079 29669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:39.468200 29669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:39.468309 29669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:22:39.468396 29669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:22:39.477964 29669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:39.479363 29669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:39.480836 29669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:39.492838 29677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:39.492913 29674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:39.492941 29675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:39.493739 29669 server_base.cc:1034] running on GCE node
I20260809 20:22:39.494509 29669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:39.496022 29669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:39.497242 29669 hybrid_clock.cc:648] HybridClock initialized: now 1786306959497210 us; error 58 us; skew 500 ppm
I20260809 20:22:39.497571 29669 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:39 dist-test-slave-6k22 krb5kdc[29654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306959, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:39.500864 29669 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 20:22:39.502372 29669 webserver.cc:460] Webserver started at http://127.0.0.1:44349/ using document root <none> and password file <none>
I20260809 20:22:39.503054 29669 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:39.503201 29669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:39.503502 29669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:39.506474 29669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "eb7228e18ed04be781fa71faff5d066f"
format_stamp: "Formatted at 2026-08-09 20:22:39 on dist-test-slave-6k22"
server_key: "ae07a46750f84f2a3edbcee6b750e5ee"
server_key_iv: "b634ce62fe71e15a401ade11acba114d"
server_key_version: "encryptionkey@0"
I20260809 20:22:39.507232 29669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "eb7228e18ed04be781fa71faff5d066f"
format_stamp: "Formatted at 2026-08-09 20:22:39 on dist-test-slave-6k22"
server_key: "ae07a46750f84f2a3edbcee6b750e5ee"
server_key_iv: "b634ce62fe71e15a401ade11acba114d"
server_key_version: "encryptionkey@0"
I20260809 20:22:39.512430 29669 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 20:22:39.515959 29685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:39.517197 29669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 20:22:39.517382 29669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "eb7228e18ed04be781fa71faff5d066f"
format_stamp: "Formatted at 2026-08-09 20:22:39 on dist-test-slave-6k22"
server_key: "ae07a46750f84f2a3edbcee6b750e5ee"
server_key_iv: "b634ce62fe71e15a401ade11acba114d"
server_key_version: "encryptionkey@0"
I20260809 20:22:39.517567 29669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:39.549345 29669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:39.553684 29669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:39.553977 29669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:39.572490 29669 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38639
I20260809 20:22:39.572535 29746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38639 every 8 connection(s)
I20260809 20:22:39.573899 29669 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:22:39.576828 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 29669
I20260809 20:22:39.577082 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:22:39.577558 5364 external_mini_cluster.cc:1428] Setting key 842d8e4d7ad2650014f1e4cc9d7acfc4
I20260809 20:22:39.577723 29747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:39.591935 29747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb7228e18ed04be781fa71faff5d066f: Bootstrap starting.
Aug 09 20:22:39 dist-test-slave-6k22 krb5kdc[29654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 20:22:39.595144 29747 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb7228e18ed04be781fa71faff5d066f: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:39.596357 29747 log.cc:824] T 00000000000000000000000000000000 P eb7228e18ed04be781fa71faff5d066f: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:39.599110 29747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb7228e18ed04be781fa71faff5d066f: No bootstrap required, opened a new log
I20260809 20:22:39.600759 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:39.579332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:54209 (local address 127.0.0.1:38639)
0809 20:22:39.580160 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:39.580173 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:39.580218 (+ 45us) server_negotiation.cc:407] Connection header received
0809 20:22:39.582255 (+ 2037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:39.582275 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:39.582651 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:39.583001 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:39.584865 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:39.586181 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:39.587982 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:39.588636 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:39.588794 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:39.593556 (+ 4762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:39.593563 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:39.593575 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:39.593609 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:39.596645 (+ 3036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:39.597448 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:39.597453 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:39.597458 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:39.597628 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:39.598117 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:39.598120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:39.598123 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:39.598618 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:39.598955 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:39.599436 (+ 481us) server_negotiation.cc:299] Negotiation successful
0809 20:22:39.599788 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":409,"threads_started":1}
I20260809 20:22:39.604317 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:22:39.607007 29747 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb7228e18ed04be781fa71faff5d066f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb7228e18ed04be781fa71faff5d066f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38639 } }
I20260809 20:22:39.607483 29747 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb7228e18ed04be781fa71faff5d066f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:39.607627 29747 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb7228e18ed04be781fa71faff5d066f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb7228e18ed04be781fa71faff5d066f, State: Initialized, Role: FOLLOWER
I20260809 20:22:39.608204 29747 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb7228e18ed04be781fa71faff5d066f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb7228e18ed04be781fa71faff5d066f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38639 } }
I20260809 20:22:39.608520 29747 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb7228e18ed04be781fa71faff5d066f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:22:39.608677 29747 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb7228e18ed04be781fa71faff5d066f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:22:39.608968 29747 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb7228e18ed04be781fa71faff5d066f [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:39.612574 29747 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb7228e18ed04be781fa71faff5d066f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb7228e18ed04be781fa71faff5d066f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38639 } }
I20260809 20:22:39.613054 29747 leader_election.cc:302] T 00000000000000000000000000000000 P eb7228e18ed04be781fa71faff5d066f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb7228e18ed04be781fa71faff5d066f; no voters:
I20260809 20:22:39.613641 29747 leader_election.cc:288] T 00000000000000000000000000000000 P eb7228e18ed04be781fa71faff5d066f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:22:39.613813 29760 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb7228e18ed04be781fa71faff5d066f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:39.615106 29760 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb7228e18ed04be781fa71faff5d066f [term 1 LEADER]: Becoming Leader. State: Replica: eb7228e18ed04be781fa71faff5d066f, State: Running, Role: LEADER
I20260809 20:22:39.615715 29760 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb7228e18ed04be781fa71faff5d066f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb7228e18ed04be781fa71faff5d066f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38639 } }
I20260809 20:22:39.616180 29747 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb7228e18ed04be781fa71faff5d066f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:22:39.618928 29763 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb7228e18ed04be781fa71faff5d066f [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb7228e18ed04be781fa71faff5d066f. Latest consensus state: current_term: 1 leader_uuid: "eb7228e18ed04be781fa71faff5d066f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb7228e18ed04be781fa71faff5d066f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38639 } } }
I20260809 20:22:39.619346 29763 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb7228e18ed04be781fa71faff5d066f [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:39.619833 29761 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb7228e18ed04be781fa71faff5d066f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb7228e18ed04be781fa71faff5d066f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb7228e18ed04be781fa71faff5d066f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38639 } } }
I20260809 20:22:39.620231 29761 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb7228e18ed04be781fa71faff5d066f [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:39.624203 29773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:22:39.629778 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:39.608624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:38469 (local address 127.0.0.1:38639)
0809 20:22:39.608826 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:39.608831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:39.612106 (+ 3275us) server_negotiation.cc:407] Connection header received
0809 20:22:39.612292 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:39.612303 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:39.612360 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:39.612460 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:39.614132 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:39.615769 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:39.617688 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:39.618482 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:39.618635 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:39.623076 (+ 4441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:39.623082 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:39.623087 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:39.623120 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:39.626038 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:39.626931 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:39.626938 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:39.626943 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:39.627095 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:39.627739 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:39.627744 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:39.627747 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:39.628227 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:39.628505 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:39.629226 (+ 721us) server_negotiation.cc:299] Negotiation successful
0809 20:22:39.629421 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 20:22:39.631011 29773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:22:39.645906 29773 catalog_manager.cc:1347] Generated new cluster ID: eb9c8d4609b44cefa8441637ea4f8e3f
I20260809 20:22:39.646047 29773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:22:39.667448 29773 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:22:39.669003 29773 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:22:39.675253 29773 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb7228e18ed04be781fa71faff5d066f: Generated new TSK 0
I20260809 20:22:39.676237 29773 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:22:39.682754 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 29669
I20260809 20:22:39.683849 29742 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:39.776053 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 29669
W20260809 20:22:39.793454 29652 connection.cc:535] client connection to 127.0.0.1:38639 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38639): Connection reset by peer (error 104)
2026-08-09T20:22:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2156 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:22:39 dist-test-slave-6k22 krb5kdc[29786](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:22:39 dist-test-slave-6k22 krb5kdc[29786](info): set up 2 sockets
Aug 09 20:22:39 dist-test-slave-6k22 krb5kdc[29786](info): commencing operation
krb5kdc: starting...
W20260809 20:22:40.984170 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.001s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:22:41 dist-test-slave-6k22 krb5kdc[29786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:22:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:22:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 20:22:41.279335 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38201
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:41.613152 29804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:41.613425 29804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:41.613615 29804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:41.620950 29804 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:22:41.621079 29804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:41.621129 29804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:41.621237 29804 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:22:41.621302 29804 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:22:41.631027 29804 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:41.632344 29804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:41.633674 29804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:41.645809 29809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:41.645848 29815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:41.645823 29810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:41.646834 29804 server_base.cc:1034] running on GCE node
I20260809 20:22:41.647660 29804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:41.648986 29804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:41.650214 29804 hybrid_clock.cc:648] HybridClock initialized: now 1786306961650166 us; error 49 us; skew 500 ppm
I20260809 20:22:41.650554 29804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:41 dist-test-slave-6k22 krb5kdc[29786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306961, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:41.653625 29804 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 20:22:41.655182 29804 webserver.cc:460] Webserver started at http://127.0.0.1:33205/ using document root <none> and password file <none>
I20260809 20:22:41.655938 29804 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:41.656085 29804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:41.656406 29804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:41.659349 29804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "7fc08b3b1c3f43e78e9917ad7d184c34"
format_stamp: "Formatted at 2026-08-09 20:22:41 on dist-test-slave-6k22"
server_key: "c00147f188efca7e5f59c429ad25d3d0"
server_key_iv: "b6abef661850140e6b422d0058711a29"
server_key_version: "encryptionkey@0"
I20260809 20:22:41.660193 29804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "7fc08b3b1c3f43e78e9917ad7d184c34"
format_stamp: "Formatted at 2026-08-09 20:22:41 on dist-test-slave-6k22"
server_key: "c00147f188efca7e5f59c429ad25d3d0"
server_key_iv: "b6abef661850140e6b422d0058711a29"
server_key_version: "encryptionkey@0"
I20260809 20:22:41.665297 29804 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 20:22:41.668687 29825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:41.669957 29804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:22:41.670140 29804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "7fc08b3b1c3f43e78e9917ad7d184c34"
format_stamp: "Formatted at 2026-08-09 20:22:41 on dist-test-slave-6k22"
server_key: "c00147f188efca7e5f59c429ad25d3d0"
server_key_iv: "b6abef661850140e6b422d0058711a29"
server_key_version: "encryptionkey@0"
I20260809 20:22:41.670332 29804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:41.692618 29804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:41.696970 29804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:41.697258 29804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:41.716331 29804 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38201
I20260809 20:22:41.716377 29886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38201 every 8 connection(s)
I20260809 20:22:41.717690 29804 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:22:41.718115 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 29804
I20260809 20:22:41.718329 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:22:41.719041 5364 external_mini_cluster.cc:1428] Setting key ea2b6ddba2c5e0547573ee03870ff9fa
I20260809 20:22:41.723222 29887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:41.733548 29891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:41.722401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:37661 (local address 127.0.0.1:38201)
0809 20:22:41.723181 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:41.723192 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:41.723234 (+ 42us) server_negotiation.cc:407] Connection header received
0809 20:22:41.724955 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:41.724975 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:41.725362 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:41.725787 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:41.726322 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:41.727679 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:41.728991 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:41.729617 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:41.729826 (+ 209us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:41.731237 (+ 1411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:41.731244 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:41.731257 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:41.731865 (+ 608us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:41.731936 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:41.731959 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:41.732133 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:41.732415 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 20:22:41.732617 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":359,"threads_started":1}
I20260809 20:22:41.733788 29887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fc08b3b1c3f43e78e9917ad7d184c34: Bootstrap starting.
I20260809 20:22:41.737098 29887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7fc08b3b1c3f43e78e9917ad7d184c34: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:41.737353 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:22:41.738222 29887 log.cc:824] T 00000000000000000000000000000000 P 7fc08b3b1c3f43e78e9917ad7d184c34: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:41.741165 29887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fc08b3b1c3f43e78e9917ad7d184c34: No bootstrap required, opened a new log
I20260809 20:22:41.749733 29891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:41.741821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:41319 (local address 127.0.0.1:38201)
0809 20:22:41.742037 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:41.742046 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:41.742521 (+ 475us) server_negotiation.cc:407] Connection header received
0809 20:22:41.742738 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:41.742741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:41.742801 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:41.742878 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:41.743670 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:41.744737 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:41.746751 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:41.747306 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:41.747436 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:41.748813 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:41.748818 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:41.748821 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:41.748839 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:41.748875 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:41.748910 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:41.749031 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:41.749245 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 20:22:41.749321 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 20:22:41.751207 29887 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7fc08b3b1c3f43e78e9917ad7d184c34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fc08b3b1c3f43e78e9917ad7d184c34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38201 } }
I20260809 20:22:41.751636 29887 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7fc08b3b1c3f43e78e9917ad7d184c34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:41.751758 29887 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7fc08b3b1c3f43e78e9917ad7d184c34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fc08b3b1c3f43e78e9917ad7d184c34, State: Initialized, Role: FOLLOWER
I20260809 20:22:41.752307 29887 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7fc08b3b1c3f43e78e9917ad7d184c34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fc08b3b1c3f43e78e9917ad7d184c34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38201 } }
I20260809 20:22:41.752542 29887 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7fc08b3b1c3f43e78e9917ad7d184c34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:22:41.752681 29887 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7fc08b3b1c3f43e78e9917ad7d184c34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:22:41.752863 29887 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7fc08b3b1c3f43e78e9917ad7d184c34 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:41.754923 29887 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7fc08b3b1c3f43e78e9917ad7d184c34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fc08b3b1c3f43e78e9917ad7d184c34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38201 } }
I20260809 20:22:41.755429 29887 leader_election.cc:302] T 00000000000000000000000000000000 P 7fc08b3b1c3f43e78e9917ad7d184c34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fc08b3b1c3f43e78e9917ad7d184c34; no voters:
I20260809 20:22:41.756031 29887 leader_election.cc:288] T 00000000000000000000000000000000 P 7fc08b3b1c3f43e78e9917ad7d184c34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:22:41.756160 29900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7fc08b3b1c3f43e78e9917ad7d184c34 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:41.758015 29900 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7fc08b3b1c3f43e78e9917ad7d184c34 [term 1 LEADER]: Becoming Leader. State: Replica: 7fc08b3b1c3f43e78e9917ad7d184c34, State: Running, Role: LEADER
I20260809 20:22:41.758162 29887 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7fc08b3b1c3f43e78e9917ad7d184c34 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:22:41.758585 29900 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7fc08b3b1c3f43e78e9917ad7d184c34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fc08b3b1c3f43e78e9917ad7d184c34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38201 } }
I20260809 20:22:41.762118 29902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fc08b3b1c3f43e78e9917ad7d184c34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7fc08b3b1c3f43e78e9917ad7d184c34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fc08b3b1c3f43e78e9917ad7d184c34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38201 } } }
I20260809 20:22:41.762552 29902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fc08b3b1c3f43e78e9917ad7d184c34 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:41.762411 29903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fc08b3b1c3f43e78e9917ad7d184c34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fc08b3b1c3f43e78e9917ad7d184c34. Latest consensus state: current_term: 1 leader_uuid: "7fc08b3b1c3f43e78e9917ad7d184c34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fc08b3b1c3f43e78e9917ad7d184c34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38201 } } }
I20260809 20:22:41.762688 29903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fc08b3b1c3f43e78e9917ad7d184c34 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:41.764256 29911 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:22:41.768859 29911 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:22:41.777069 29911 catalog_manager.cc:1347] Generated new cluster ID: f826ac7343bb4787963362fdb76eb68b
I20260809 20:22:41.777202 29911 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:22:41.800374 29911 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:22:41.801597 29911 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:22:41.815904 29911 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7fc08b3b1c3f43e78e9917ad7d184c34: Generated new TSK 0
I20260809 20:22:41.817029 29911 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:22:41.831645 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 29804
W20260809 20:22:41.832813 29882 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.223:37661
W20260809 20:22:41.833609 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 29804: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 20:22:41.833763 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 29804
W20260809 20:22:41.855706 29783 connection.cc:535] client connection to 127.0.0.1:38201 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38201): Connection reset by peer (error 104)
2026-08-09T20:22:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2062 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:22:41 dist-test-slave-6k22 krb5kdc[29924](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:22:41 dist-test-slave-6k22 krb5kdc[29924](info): set up 2 sockets
Aug 09 20:22:41 dist-test-slave-6k22 krb5kdc[29924](info): commencing operation
krb5kdc: starting...
W20260809 20:22:43.050561 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:22:43 dist-test-slave-6k22 krb5kdc[29924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:22:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:22:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 20:22:43.348474 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41845
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:43.683319 29938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:43.683682 29938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:43.683799 29938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:43.690788 29938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:22:43.690887 29938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:43.690934 29938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:43.690991 29938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:22:43.691037 29938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:22:43.700601 29938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:43.701769 29938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:43.702989 29938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:43.715229 29947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:43.715340 29944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:43.715363 29945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:43.721016 29938 server_base.cc:1034] running on GCE node
I20260809 20:22:43.721890 29938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:43.723302 29938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:43.724531 29938 hybrid_clock.cc:648] HybridClock initialized: now 1786306963724472 us; error 74 us; skew 500 ppm
I20260809 20:22:43.724891 29938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:43 dist-test-slave-6k22 krb5kdc[29924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306963, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:43.729055 29938 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 20:22:43.730537 29938 webserver.cc:460] Webserver started at http://127.0.0.1:43615/ using document root <none> and password file <none>
I20260809 20:22:43.731280 29938 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:43.731427 29938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:43.731786 29938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:43.734823 29938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "6da7c189509745f49c2a29585a45607b"
format_stamp: "Formatted at 2026-08-09 20:22:43 on dist-test-slave-6k22"
server_key: "510d050399cbf626bb11b866efc3476b"
server_key_iv: "aacec2e1c560ec289fcd3bb4856307d6"
server_key_version: "encryptionkey@0"
I20260809 20:22:43.735708 29938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "6da7c189509745f49c2a29585a45607b"
format_stamp: "Formatted at 2026-08-09 20:22:43 on dist-test-slave-6k22"
server_key: "510d050399cbf626bb11b866efc3476b"
server_key_iv: "aacec2e1c560ec289fcd3bb4856307d6"
server_key_version: "encryptionkey@0"
I20260809 20:22:43.740769 29938 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 20:22:43.744202 29956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:43.745463 29938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:22:43.745653 29938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "6da7c189509745f49c2a29585a45607b"
format_stamp: "Formatted at 2026-08-09 20:22:43 on dist-test-slave-6k22"
server_key: "510d050399cbf626bb11b866efc3476b"
server_key_iv: "aacec2e1c560ec289fcd3bb4856307d6"
server_key_version: "encryptionkey@0"
I20260809 20:22:43.745849 29938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:43.751441 29938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:43.755720 29938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:43.756018 29938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:43.774617 29938 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41845
I20260809 20:22:43.774617 30016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41845 every 8 connection(s)
I20260809 20:22:43.776117 29938 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:22:43.776566 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 29938
I20260809 20:22:43.776777 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:22:43.777383 5364 external_mini_cluster.cc:1428] Setting key 7b272f29b3e1dc0c913b924cc5e96d41
I20260809 20:22:43.780877 30017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:43.786054 30020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:43.779195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:43497 (local address 127.0.0.1:41845)
0809 20:22:43.779987 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:43.780001 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:43.780049 (+ 48us) server_negotiation.cc:407] Connection header received
0809 20:22:43.782284 (+ 2235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:43.782327 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:43.782755 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:43.783136 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:43.784153 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:43.784160 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:43.784175 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:43.784198 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:43.784296 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:43.784321 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:43.784543 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:43.784679 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 20:22:43.784854 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":345,"threads_started":1}
I20260809 20:22:43.789749 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:22:43.794265 30017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6da7c189509745f49c2a29585a45607b: Bootstrap starting.
I20260809 20:22:43.797502 30020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:43.794982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:45253 (local address 127.0.0.1:41845)
0809 20:22:43.795146 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:43.795151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:43.795683 (+ 532us) server_negotiation.cc:407] Connection header received
0809 20:22:43.795829 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:43.795834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:43.795908 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:43.796035 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:43.796918 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:43.796923 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:43.796927 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:43.796943 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:43.796988 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:43.797007 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:43.797133 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:43.797197 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 20:22:43.797257 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 20:22:43.797977 30017 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6da7c189509745f49c2a29585a45607b: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:43.799013 30017 log.cc:824] T 00000000000000000000000000000000 P 6da7c189509745f49c2a29585a45607b: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:43.801507 30017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6da7c189509745f49c2a29585a45607b: No bootstrap required, opened a new log
I20260809 20:22:43.808320 30017 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6da7c189509745f49c2a29585a45607b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6da7c189509745f49c2a29585a45607b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41845 } }
I20260809 20:22:43.808701 30017 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6da7c189509745f49c2a29585a45607b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:43.808818 30017 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6da7c189509745f49c2a29585a45607b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6da7c189509745f49c2a29585a45607b, State: Initialized, Role: FOLLOWER
I20260809 20:22:43.809345 30017 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6da7c189509745f49c2a29585a45607b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6da7c189509745f49c2a29585a45607b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41845 } }
I20260809 20:22:43.809581 30017 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6da7c189509745f49c2a29585a45607b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:22:43.809743 30017 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6da7c189509745f49c2a29585a45607b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:22:43.809911 30017 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6da7c189509745f49c2a29585a45607b [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:43.811976 30017 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6da7c189509745f49c2a29585a45607b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6da7c189509745f49c2a29585a45607b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41845 } }
I20260809 20:22:43.812427 30017 leader_election.cc:302] T 00000000000000000000000000000000 P 6da7c189509745f49c2a29585a45607b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6da7c189509745f49c2a29585a45607b; no voters:
I20260809 20:22:43.812999 30017 leader_election.cc:288] T 00000000000000000000000000000000 P 6da7c189509745f49c2a29585a45607b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:22:43.813124 30030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6da7c189509745f49c2a29585a45607b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:43.813832 30030 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6da7c189509745f49c2a29585a45607b [term 1 LEADER]: Becoming Leader. State: Replica: 6da7c189509745f49c2a29585a45607b, State: Running, Role: LEADER
I20260809 20:22:43.814317 30030 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6da7c189509745f49c2a29585a45607b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6da7c189509745f49c2a29585a45607b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41845 } }
I20260809 20:22:43.814996 30017 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6da7c189509745f49c2a29585a45607b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:22:43.817214 30032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6da7c189509745f49c2a29585a45607b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6da7c189509745f49c2a29585a45607b. Latest consensus state: current_term: 1 leader_uuid: "6da7c189509745f49c2a29585a45607b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6da7c189509745f49c2a29585a45607b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41845 } } }
I20260809 20:22:43.817687 30032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6da7c189509745f49c2a29585a45607b [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:43.824618 30031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6da7c189509745f49c2a29585a45607b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6da7c189509745f49c2a29585a45607b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6da7c189509745f49c2a29585a45607b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41845 } } }
I20260809 20:22:43.825392 30031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6da7c189509745f49c2a29585a45607b [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:43.825606 30038 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:22:43.830570 30038 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:22:43.839838 30038 catalog_manager.cc:1347] Generated new cluster ID: 10c23b7cf4594e629ab8fa565b23d83a
I20260809 20:22:43.839962 30038 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:22:43.857676 30038 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:22:43.858081 30038 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:22:43.876335 30038 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6da7c189509745f49c2a29585a45607b: Generated new TSK 0
I20260809 20:22:43.877125 30038 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:22:43.885516 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 29938
W20260809 20:22:43.886780 30010 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.223:43497
W20260809 20:22:43.887492 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 29938: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 20:22:43.887697 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 29938
2026-08-09T20:22:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2052 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:22:44 dist-test-slave-6k22 krb5kdc[30053](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:22:44 dist-test-slave-6k22 krb5kdc[30053](info): set up 2 sockets
Aug 09 20:22:44 dist-test-slave-6k22 krb5kdc[30053](info): commencing operation
krb5kdc: starting...
W20260809 20:22:45.110080 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.001s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:22:45 dist-test-slave-6k22 krb5kdc[30053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:22:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:22:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 20:22:45.411150 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36375
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:45.742116 30067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:45.742427 30067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:45.742594 30067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:45.749873 30067 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:22:45.750032 30067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:45.750133 30067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:45.750236 30067 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:22:45.750320 30067 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:22:45.759876 30067 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36375
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:45.761126 30067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:45.762432 30067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:45.774386 30077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:45.774462 30073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:45.774351 30074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:45.775193 30067 server_base.cc:1034] running on GCE node
I20260809 20:22:45.776038 30067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:45.777333 30067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:45.778533 30067 hybrid_clock.cc:648] HybridClock initialized: now 1786306965778490 us; error 35 us; skew 500 ppm
I20260809 20:22:45.778872 30067 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:45 dist-test-slave-6k22 krb5kdc[30053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306965, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:45.781949 30067 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 20:22:45.783408 30067 webserver.cc:460] Webserver started at http://127.0.0.1:32857/ using document root <none> and password file <none>
I20260809 20:22:45.784126 30067 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:45.784263 30067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:45.784596 30067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:45.787429 30067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "e012bdb421a640b2abb98c4c31c774da"
format_stamp: "Formatted at 2026-08-09 20:22:45 on dist-test-slave-6k22"
server_key: "5c2f3d72ad21c3088dbac643269fa947"
server_key_iv: "9925d4fd8b3cf48ffb6552a78618cc7d"
server_key_version: "encryptionkey@0"
I20260809 20:22:45.788271 30067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "e012bdb421a640b2abb98c4c31c774da"
format_stamp: "Formatted at 2026-08-09 20:22:45 on dist-test-slave-6k22"
server_key: "5c2f3d72ad21c3088dbac643269fa947"
server_key_iv: "9925d4fd8b3cf48ffb6552a78618cc7d"
server_key_version: "encryptionkey@0"
I20260809 20:22:45.792958 30067 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 20:22:45.796207 30084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:45.797500 30067 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 20:22:45.797681 30067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "e012bdb421a640b2abb98c4c31c774da"
format_stamp: "Formatted at 2026-08-09 20:22:45 on dist-test-slave-6k22"
server_key: "5c2f3d72ad21c3088dbac643269fa947"
server_key_iv: "9925d4fd8b3cf48ffb6552a78618cc7d"
server_key_version: "encryptionkey@0"
I20260809 20:22:45.797894 30067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:45.814419 30067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:45.818198 30067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:45.818493 30067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:45.836493 30067 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36375
I20260809 20:22:45.836526 30150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36375 every 8 connection(s)
I20260809 20:22:45.837819 30067 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:22:45.841420 30151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:45.841486 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 30067
I20260809 20:22:45.841679 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:22:45.842150 5364 external_mini_cluster.cc:1428] Setting key 76051758870be922a790ec690cb5836d
I20260809 20:22:45.855742 30151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e012bdb421a640b2abb98c4c31c774da: Bootstrap starting.
Aug 09 20:22:45 dist-test-slave-6k22 krb5kdc[30053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 20:22:45.858824 30151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e012bdb421a640b2abb98c4c31c774da: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:45.859956 30151 log.cc:824] T 00000000000000000000000000000000 P e012bdb421a640b2abb98c4c31c774da: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:45.862648 30151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e012bdb421a640b2abb98c4c31c774da: No bootstrap required, opened a new log
I20260809 20:22:45.865020 30154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:45.843865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:39691 (local address 127.0.0.1:36375)
0809 20:22:45.844696 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:45.844706 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:45.844748 (+ 42us) server_negotiation.cc:407] Connection header received
0809 20:22:45.846262 (+ 1514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:45.846281 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:45.846668 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:45.846973 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:45.848847 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:45.850101 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:45.851869 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:45.852558 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:45.852685 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:45.857550 (+ 4865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:45.857557 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:45.857569 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:45.857606 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:45.860824 (+ 3218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:45.861636 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:45.861642 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:45.861648 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:45.861818 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:45.862307 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:45.862310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:45.862312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:45.862854 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:45.863192 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:45.863742 (+ 550us) server_negotiation.cc:299] Negotiation successful
0809 20:22:45.864067 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":429,"threads_started":1}
I20260809 20:22:45.868813 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:22:45.870695 30151 raft_consensus.cc:348] T 00000000000000000000000000000000 P e012bdb421a640b2abb98c4c31c774da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e012bdb421a640b2abb98c4c31c774da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36375 } }
I20260809 20:22:45.871042 30151 raft_consensus.cc:374] T 00000000000000000000000000000000 P e012bdb421a640b2abb98c4c31c774da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:45.871160 30151 raft_consensus.cc:729] T 00000000000000000000000000000000 P e012bdb421a640b2abb98c4c31c774da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e012bdb421a640b2abb98c4c31c774da, State: Initialized, Role: FOLLOWER
I20260809 20:22:45.871797 30151 consensus_queue.cc:260] T 00000000000000000000000000000000 P e012bdb421a640b2abb98c4c31c774da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e012bdb421a640b2abb98c4c31c774da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36375 } }
I20260809 20:22:45.872044 30151 raft_consensus.cc:388] T 00000000000000000000000000000000 P e012bdb421a640b2abb98c4c31c774da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:22:45.872197 30151 raft_consensus.cc:482] T 00000000000000000000000000000000 P e012bdb421a640b2abb98c4c31c774da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:22:45.872385 30151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e012bdb421a640b2abb98c4c31c774da [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:45.874929 30151 raft_consensus.cc:504] T 00000000000000000000000000000000 P e012bdb421a640b2abb98c4c31c774da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e012bdb421a640b2abb98c4c31c774da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36375 } }
I20260809 20:22:45.875423 30151 leader_election.cc:302] T 00000000000000000000000000000000 P e012bdb421a640b2abb98c4c31c774da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e012bdb421a640b2abb98c4c31c774da; no voters:
I20260809 20:22:45.876135 30151 leader_election.cc:288] T 00000000000000000000000000000000 P e012bdb421a640b2abb98c4c31c774da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:22:45.876236 30162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e012bdb421a640b2abb98c4c31c774da [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:45.878576 30151 sys_catalog.cc:564] T 00000000000000000000000000000000 P e012bdb421a640b2abb98c4c31c774da [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:22:45.879843 30162 raft_consensus.cc:686] T 00000000000000000000000000000000 P e012bdb421a640b2abb98c4c31c774da [term 1 LEADER]: Becoming Leader. State: Replica: e012bdb421a640b2abb98c4c31c774da, State: Running, Role: LEADER
I20260809 20:22:45.880560 30162 consensus_queue.cc:237] T 00000000000000000000000000000000 P e012bdb421a640b2abb98c4c31c774da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e012bdb421a640b2abb98c4c31c774da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36375 } }
I20260809 20:22:45.884351 30163 sys_catalog.cc:455] T 00000000000000000000000000000000 P e012bdb421a640b2abb98c4c31c774da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e012bdb421a640b2abb98c4c31c774da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e012bdb421a640b2abb98c4c31c774da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36375 } } }
I20260809 20:22:45.884752 30163 sys_catalog.cc:458] T 00000000000000000000000000000000 P e012bdb421a640b2abb98c4c31c774da [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:45.885071 30164 sys_catalog.cc:455] T 00000000000000000000000000000000 P e012bdb421a640b2abb98c4c31c774da [sys.catalog]: SysCatalogTable state changed. Reason: New leader e012bdb421a640b2abb98c4c31c774da. Latest consensus state: current_term: 1 leader_uuid: "e012bdb421a640b2abb98c4c31c774da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e012bdb421a640b2abb98c4c31c774da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36375 } } }
I20260809 20:22:45.885403 30164 sys_catalog.cc:458] T 00000000000000000000000000000000 P e012bdb421a640b2abb98c4c31c774da [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:45.891211 30154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:45.873201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:48143 (local address 127.0.0.1:36375)
0809 20:22:45.873384 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:45.873391 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:45.873731 (+ 340us) server_negotiation.cc:407] Connection header received
0809 20:22:45.873915 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:45.873920 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:45.873995 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:45.874098 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:45.875537 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:45.876589 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:45.878622 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:45.879156 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:45.879270 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:45.881913 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:45.881921 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:45.881929 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:45.881956 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:45.886242 (+ 4286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:45.887262 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:45.887267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:45.887270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:45.887415 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:45.888104 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:45.888108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:45.888111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:45.888547 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:45.890274 (+ 1727us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:45.890485 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 20:22:45.890709 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 20:22:45.896608 30171 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:22:45.901751 30171 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:22:45.910212 30171 catalog_manager.cc:1347] Generated new cluster ID: 7739362976cf49eea65cde7416e7d0e7
I20260809 20:22:45.910337 30171 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:22:45.949301 30171 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:22:45.950621 30171 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:22:45.961118 30171 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e012bdb421a640b2abb98c4c31c774da: Generated new TSK 0
I20260809 20:22:45.961897 30171 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:22:45.972909 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 30067
I20260809 20:22:45.973892 30146 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:46.062863 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 30067
W20260809 20:22:46.085122 30051 connection.cc:535] client connection to 127.0.0.1:36375 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36375): Connection reset by peer (error 104)
2026-08-09T20:22:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2176 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:22:46 dist-test-slave-6k22 krb5kdc[30187](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:22:46 dist-test-slave-6k22 krb5kdc[30187](info): set up 2 sockets
Aug 09 20:22:46 dist-test-slave-6k22 krb5kdc[30187](info): commencing operation
krb5kdc: starting...
W20260809 20:22:47.261003 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:22:47 dist-test-slave-6k22 krb5kdc[30187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:22:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:22:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 20:22:47.558823 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42253
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:47.890553 30201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:47.890832 30201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:47.890955 30201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:47.898211 30201 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:22:47.898310 30201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:47.898444 30201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:47.898550 30201 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:22:47.898636 30201 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:22:47.908176 30201 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42253
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:47.909392 30201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:47.910701 30201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:47.922500 30212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:47.922591 30210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:47.922567 30209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:47.923892 30201 server_base.cc:1034] running on GCE node
I20260809 20:22:47.924705 30201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:47.926079 30201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:47.927397 30201 hybrid_clock.cc:648] HybridClock initialized: now 1786306967927355 us; error 67 us; skew 500 ppm
I20260809 20:22:47.927820 30201 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:47 dist-test-slave-6k22 krb5kdc[30187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306967, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:47.930953 30201 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 20:22:47.932323 30201 webserver.cc:460] Webserver started at http://127.0.0.1:45875/ using document root <none> and password file <none>
I20260809 20:22:47.932911 30201 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:47.933010 30201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:47.933245 30201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:47.936172 30201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "3a8d9b17d4da4050853dcf8dbb564d5d"
format_stamp: "Formatted at 2026-08-09 20:22:47 on dist-test-slave-6k22"
server_key: "a00864c3ce21865682372c168415da76"
server_key_iv: "67e42f8964dee283e36e4f78a4fe2aad"
server_key_version: "encryptionkey@0"
I20260809 20:22:47.936954 30201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "3a8d9b17d4da4050853dcf8dbb564d5d"
format_stamp: "Formatted at 2026-08-09 20:22:47 on dist-test-slave-6k22"
server_key: "a00864c3ce21865682372c168415da76"
server_key_iv: "67e42f8964dee283e36e4f78a4fe2aad"
server_key_version: "encryptionkey@0"
I20260809 20:22:47.941910 30201 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 20:22:47.945240 30222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:47.946553 30201 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 20:22:47.946746 30201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "3a8d9b17d4da4050853dcf8dbb564d5d"
format_stamp: "Formatted at 2026-08-09 20:22:47 on dist-test-slave-6k22"
server_key: "a00864c3ce21865682372c168415da76"
server_key_iv: "67e42f8964dee283e36e4f78a4fe2aad"
server_key_version: "encryptionkey@0"
I20260809 20:22:47.946930 30201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:47.962663 30201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:47.966920 30201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:47.967190 30201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:47.985546 30201 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42253
I20260809 20:22:47.985592 30284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42253 every 8 connection(s)
I20260809 20:22:47.986927 30201 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:22:47.988009 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 30201
I20260809 20:22:47.988194 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:22:47.988703 5364 external_mini_cluster.cc:1428] Setting key 8a224ee9e40bac7ca81d063cae3ff05c
I20260809 20:22:47.991243 30285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:48.002221 30288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:47.990451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:32975 (local address 127.0.0.1:42253)
0809 20:22:47.991234 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:47.991247 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:47.991301 (+ 54us) server_negotiation.cc:407] Connection header received
0809 20:22:47.992946 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:47.992965 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:47.993329 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:47.993643 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:47.994263 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:47.995615 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:47.997588 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:47.998236 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:47.998358 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:47.999766 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:47.999773 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:47.999786 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:48.000339 (+ 553us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:48.000413 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:48.000445 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:48.000564 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:48.000896 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 20:22:48.001098 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":374,"threads_started":1}
I20260809 20:22:48.003926 30285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a8d9b17d4da4050853dcf8dbb564d5d: Bootstrap starting.
I20260809 20:22:48.006201 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:22:48.007323 30285 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a8d9b17d4da4050853dcf8dbb564d5d: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:48.008384 30285 log.cc:824] T 00000000000000000000000000000000 P 3a8d9b17d4da4050853dcf8dbb564d5d: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:48.011596 30285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a8d9b17d4da4050853dcf8dbb564d5d: No bootstrap required, opened a new log
I20260809 20:22:48.018160 30288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:48.010817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:37637 (local address 127.0.0.1:42253)
0809 20:22:48.011023 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:48.011029 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:48.011465 (+ 436us) server_negotiation.cc:407] Connection header received
0809 20:22:48.011720 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:48.011751 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:48.011829 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:48.011909 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:48.012626 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:48.013392 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:48.015422 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:48.015970 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:48.016066 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:48.017317 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:48.017321 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:48.017324 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:48.017343 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:48.017411 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:48.017428 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:48.017517 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:48.017747 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 20:22:48.017818 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 20:22:48.021135 30285 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a8d9b17d4da4050853dcf8dbb564d5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a8d9b17d4da4050853dcf8dbb564d5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42253 } }
I20260809 20:22:48.021528 30285 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a8d9b17d4da4050853dcf8dbb564d5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:48.021651 30285 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a8d9b17d4da4050853dcf8dbb564d5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a8d9b17d4da4050853dcf8dbb564d5d, State: Initialized, Role: FOLLOWER
I20260809 20:22:48.022221 30285 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a8d9b17d4da4050853dcf8dbb564d5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a8d9b17d4da4050853dcf8dbb564d5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42253 } }
I20260809 20:22:48.022475 30285 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a8d9b17d4da4050853dcf8dbb564d5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:22:48.022625 30285 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a8d9b17d4da4050853dcf8dbb564d5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:22:48.022791 30285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a8d9b17d4da4050853dcf8dbb564d5d [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:48.025069 30285 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a8d9b17d4da4050853dcf8dbb564d5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a8d9b17d4da4050853dcf8dbb564d5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42253 } }
I20260809 20:22:48.025552 30285 leader_election.cc:302] T 00000000000000000000000000000000 P 3a8d9b17d4da4050853dcf8dbb564d5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a8d9b17d4da4050853dcf8dbb564d5d; no voters:
I20260809 20:22:48.026131 30285 leader_election.cc:288] T 00000000000000000000000000000000 P 3a8d9b17d4da4050853dcf8dbb564d5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:22:48.026262 30296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a8d9b17d4da4050853dcf8dbb564d5d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:48.027045 30296 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a8d9b17d4da4050853dcf8dbb564d5d [term 1 LEADER]: Becoming Leader. State: Replica: 3a8d9b17d4da4050853dcf8dbb564d5d, State: Running, Role: LEADER
I20260809 20:22:48.027603 30296 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a8d9b17d4da4050853dcf8dbb564d5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a8d9b17d4da4050853dcf8dbb564d5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42253 } }
I20260809 20:22:48.028391 30285 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a8d9b17d4da4050853dcf8dbb564d5d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:22:48.035915 30298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a8d9b17d4da4050853dcf8dbb564d5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a8d9b17d4da4050853dcf8dbb564d5d. Latest consensus state: current_term: 1 leader_uuid: "3a8d9b17d4da4050853dcf8dbb564d5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a8d9b17d4da4050853dcf8dbb564d5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42253 } } }
I20260809 20:22:48.036394 30298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a8d9b17d4da4050853dcf8dbb564d5d [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:48.035904 30297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a8d9b17d4da4050853dcf8dbb564d5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a8d9b17d4da4050853dcf8dbb564d5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a8d9b17d4da4050853dcf8dbb564d5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42253 } } }
I20260809 20:22:48.036573 30297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a8d9b17d4da4050853dcf8dbb564d5d [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:48.038691 30310 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:22:48.042686 30310 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:22:48.051805 30310 catalog_manager.cc:1347] Generated new cluster ID: 636bb25fb1b34309b10e1d6e367bdbdf
I20260809 20:22:48.051918 30310 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:22:48.076246 30310 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:22:48.078047 30310 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:22:48.087452 30310 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a8d9b17d4da4050853dcf8dbb564d5d: Generated new TSK 0
I20260809 20:22:48.088284 30310 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:22:48.105448 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 30201
W20260809 20:22:48.106472 30280 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.223:32975
W20260809 20:22:48.107228 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 30201: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 20:22:48.107384 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 30201
W20260809 20:22:48.124977 30185 connection.cc:535] client connection to 127.0.0.1:42253 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42253): Connection reset by peer (error 104)
2026-08-09T20:22:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2044 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:22:48 dist-test-slave-6k22 krb5kdc[30318](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:22:48 dist-test-slave-6k22 krb5kdc[30318](info): set up 2 sockets
Aug 09 20:22:48 dist-test-slave-6k22 krb5kdc[30318](info): commencing operation
krb5kdc: starting...
W20260809 20:22:49.310354 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:22:49 dist-test-slave-6k22 krb5kdc[30318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:22:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:22:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 20:22:49.610532 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36415
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:49.947074 30335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:49.947425 30335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:49.947542 30335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:49.954581 30335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:22:49.954680 30335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:49.954726 30335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:49.954778 30335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:22:49.954823 30335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:22:49.964974 30335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36415
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:49.966164 30335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:49.967403 30335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:49.979821 30343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:49.979926 30341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:49.979938 30340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:49.981182 30335 server_base.cc:1034] running on GCE node
I20260809 20:22:49.982008 30335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:49.983408 30335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:49.984654 30335 hybrid_clock.cc:648] HybridClock initialized: now 1786306969984591 us; error 58 us; skew 500 ppm
I20260809 20:22:49.985024 30335 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:49 dist-test-slave-6k22 krb5kdc[30318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306969, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:49.988269 30335 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 20:22:49.989821 30335 webserver.cc:460] Webserver started at http://127.0.0.1:44855/ using document root <none> and password file <none>
I20260809 20:22:49.990525 30335 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:49.990674 30335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:49.990993 30335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:49.994031 30335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "05ec7941ddba4a13971534cd49eacab2"
format_stamp: "Formatted at 2026-08-09 20:22:49 on dist-test-slave-6k22"
server_key: "01a71585fece6bec1a9ec8f69cbd6b56"
server_key_iv: "c98fef0653def0a4c91672e44b893e52"
server_key_version: "encryptionkey@0"
I20260809 20:22:49.994813 30335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "05ec7941ddba4a13971534cd49eacab2"
format_stamp: "Formatted at 2026-08-09 20:22:49 on dist-test-slave-6k22"
server_key: "01a71585fece6bec1a9ec8f69cbd6b56"
server_key_iv: "c98fef0653def0a4c91672e44b893e52"
server_key_version: "encryptionkey@0"
I20260809 20:22:50.000236 30335 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 20:22:50.003747 30350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:50.004980 30335 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 20:22:50.005177 30335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "05ec7941ddba4a13971534cd49eacab2"
format_stamp: "Formatted at 2026-08-09 20:22:49 on dist-test-slave-6k22"
server_key: "01a71585fece6bec1a9ec8f69cbd6b56"
server_key_iv: "c98fef0653def0a4c91672e44b893e52"
server_key_version: "encryptionkey@0"
I20260809 20:22:50.005386 30335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:50.011376 30335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:50.015208 30335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:50.015522 30335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:50.033938 30335 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36415
I20260809 20:22:50.033977 30413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36415 every 8 connection(s)
I20260809 20:22:50.035249 30335 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:22:50.038825 30414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:50.039021 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 30335
I20260809 20:22:50.039211 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:22:50.039749 5364 external_mini_cluster.cc:1428] Setting key 2b8d3fafd4e441c630b4e2dcb697417c
I20260809 20:22:50.047897 30417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:50.041494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:41217 (local address 127.0.0.1:36415)
0809 20:22:50.042304 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:50.042315 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:50.042354 (+ 39us) server_negotiation.cc:407] Connection header received
0809 20:22:50.044582 (+ 2228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:50.044602 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:50.044919 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:50.045224 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:50.046302 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:50.046307 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:50.046320 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:50.046333 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:50.046399 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:50.046422 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:50.046580 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:50.046713 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 20:22:50.046908 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":370,"threads_started":1}
I20260809 20:22:50.051383 30414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05ec7941ddba4a13971534cd49eacab2: Bootstrap starting.
I20260809 20:22:50.051405 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:22:50.054291 30414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05ec7941ddba4a13971534cd49eacab2: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:50.055300 30414 log.cc:824] T 00000000000000000000000000000000 P 05ec7941ddba4a13971534cd49eacab2: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:50.058071 30414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05ec7941ddba4a13971534cd49eacab2: No bootstrap required, opened a new log
I20260809 20:22:50.058638 30417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:50.055834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:48069 (local address 127.0.0.1:36415)
0809 20:22:50.055994 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:50.055999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:50.056693 (+ 694us) server_negotiation.cc:407] Connection header received
0809 20:22:50.056896 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:50.056914 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:50.056958 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:50.057053 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:50.057919 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:50.057925 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:50.057929 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:50.057946 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:50.057990 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:50.058004 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:50.058127 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:50.058249 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 20:22:50.058305 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 20:22:50.065779 30414 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05ec7941ddba4a13971534cd49eacab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05ec7941ddba4a13971534cd49eacab2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36415 } }
I20260809 20:22:50.066259 30414 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05ec7941ddba4a13971534cd49eacab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:50.066380 30414 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05ec7941ddba4a13971534cd49eacab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05ec7941ddba4a13971534cd49eacab2, State: Initialized, Role: FOLLOWER
I20260809 20:22:50.066926 30414 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05ec7941ddba4a13971534cd49eacab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05ec7941ddba4a13971534cd49eacab2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36415 } }
I20260809 20:22:50.067175 30414 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05ec7941ddba4a13971534cd49eacab2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:22:50.067356 30414 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05ec7941ddba4a13971534cd49eacab2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:22:50.067523 30414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05ec7941ddba4a13971534cd49eacab2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:50.069715 30414 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05ec7941ddba4a13971534cd49eacab2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05ec7941ddba4a13971534cd49eacab2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36415 } }
I20260809 20:22:50.070183 30414 leader_election.cc:302] T 00000000000000000000000000000000 P 05ec7941ddba4a13971534cd49eacab2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05ec7941ddba4a13971534cd49eacab2; no voters:
I20260809 20:22:50.070721 30414 leader_election.cc:288] T 00000000000000000000000000000000 P 05ec7941ddba4a13971534cd49eacab2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:22:50.070889 30425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05ec7941ddba4a13971534cd49eacab2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:50.071522 30425 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05ec7941ddba4a13971534cd49eacab2 [term 1 LEADER]: Becoming Leader. State: Replica: 05ec7941ddba4a13971534cd49eacab2, State: Running, Role: LEADER
I20260809 20:22:50.072154 30425 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05ec7941ddba4a13971534cd49eacab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05ec7941ddba4a13971534cd49eacab2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36415 } }
I20260809 20:22:50.072788 30414 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05ec7941ddba4a13971534cd49eacab2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:22:50.075129 30427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05ec7941ddba4a13971534cd49eacab2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05ec7941ddba4a13971534cd49eacab2. Latest consensus state: current_term: 1 leader_uuid: "05ec7941ddba4a13971534cd49eacab2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05ec7941ddba4a13971534cd49eacab2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36415 } } }
I20260809 20:22:50.075719 30427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05ec7941ddba4a13971534cd49eacab2 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:50.075153 30426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05ec7941ddba4a13971534cd49eacab2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05ec7941ddba4a13971534cd49eacab2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05ec7941ddba4a13971534cd49eacab2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36415 } } }
I20260809 20:22:50.076059 30426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05ec7941ddba4a13971534cd49eacab2 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:50.079107 30436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:22:50.083127 30436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:22:50.091020 30436 catalog_manager.cc:1347] Generated new cluster ID: fff936667601432daf337ffea131792c
I20260809 20:22:50.091123 30436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:22:50.113762 30436 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:22:50.114212 30436 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:22:50.123239 30436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05ec7941ddba4a13971534cd49eacab2: Generated new TSK 0
I20260809 20:22:50.124192 30436 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:22:50.135939 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 30335
W20260809 20:22:50.137226 30408 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.223:41217
W20260809 20:22:50.137913 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 30335: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 20:22:50.138064 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 30335
2026-08-09T20:22:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2027 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:22:50 dist-test-slave-6k22 krb5kdc[30449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:22:50 dist-test-slave-6k22 krb5kdc[30449](info): set up 2 sockets
Aug 09 20:22:50 dist-test-slave-6k22 krb5kdc[30449](info): commencing operation
krb5kdc: starting...
W20260809 20:22:51.339076 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.008s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:22:51 dist-test-slave-6k22 krb5kdc[30449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:22:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:22:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:22:51.646711 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:35367
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:44459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:35367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:51.982139 30464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:51.982407 30464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:51.982591 30464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:51.989552 30464 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:22:51.989681 30464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:51.989804 30464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:51.989918 30464 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:22:51.989998 30464 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:22:51.999302 30464 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:44459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:35367
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:35367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:52.000581 30464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:52.001891 30464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:52.013831 30469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:52.013893 30473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:52.013967 30470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:52.014912 30464 server_base.cc:1034] running on GCE node
I20260809 20:22:52.015700 30464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:52.017148 30464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:52.018407 30464 hybrid_clock.cc:648] HybridClock initialized: now 1786306972018343 us; error 34 us; skew 500 ppm
I20260809 20:22:52.018752 30464 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:52 dist-test-slave-6k22 krb5kdc[30449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306972, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:52.021822 30464 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:22:52.023319 30464 webserver.cc:460] Webserver started at http://127.5.61.62:35637/ using document root <none> and password file <none>
I20260809 20:22:52.024057 30464 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:52.024212 30464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:52.024523 30464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:52.027639 30464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "dfb2c72cb92a423f9fc52219de0dc8c8"
format_stamp: "Formatted at 2026-08-09 20:22:52 on dist-test-slave-6k22"
server_key: "6efaf1c829d00b15243e8b97ce430f0b"
server_key_iv: "5ee77ab2bd5065ef8f324c1e5ae25fbb"
server_key_version: "encryptionkey@0"
I20260809 20:22:52.028422 30464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "dfb2c72cb92a423f9fc52219de0dc8c8"
format_stamp: "Formatted at 2026-08-09 20:22:52 on dist-test-slave-6k22"
server_key: "6efaf1c829d00b15243e8b97ce430f0b"
server_key_iv: "5ee77ab2bd5065ef8f324c1e5ae25fbb"
server_key_version: "encryptionkey@0"
I20260809 20:22:52.033425 30464 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 20:22:52.036751 30481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:52.037997 30464 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:22:52.038187 30464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "dfb2c72cb92a423f9fc52219de0dc8c8"
format_stamp: "Formatted at 2026-08-09 20:22:52 on dist-test-slave-6k22"
server_key: "6efaf1c829d00b15243e8b97ce430f0b"
server_key_iv: "5ee77ab2bd5065ef8f324c1e5ae25fbb"
server_key_version: "encryptionkey@0"
I20260809 20:22:52.038403 30464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:52.067723 30464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:52.071602 30464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:52.071902 30464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:52.090327 30464 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:35367
I20260809 20:22:52.090374 30544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:35367 every 8 connection(s)
I20260809 20:22:52.091702 30464 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:22:52.095253 30545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:52.096632 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 30464
I20260809 20:22:52.096846 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:22:52.097455 5364 external_mini_cluster.cc:1428] Setting key 44d0dbe203fa213f0e14a1bde4692521
I20260809 20:22:52.109745 30545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfb2c72cb92a423f9fc52219de0dc8c8: Bootstrap starting.
Aug 09 20:22:52 dist-test-slave-6k22 krb5kdc[30449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:22:52.112741 30545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dfb2c72cb92a423f9fc52219de0dc8c8: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:52.113763 30545 log.cc:824] T 00000000000000000000000000000000 P dfb2c72cb92a423f9fc52219de0dc8c8: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:52.116671 30545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfb2c72cb92a423f9fc52219de0dc8c8: No bootstrap required, opened a new log
I20260809 20:22:52.121057 30548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:52.099295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37153 (local address 127.5.61.62:35367)
0809 20:22:52.100255 (+ 960us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:52.100266 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:52.100308 (+ 42us) server_negotiation.cc:407] Connection header received
0809 20:22:52.101855 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:52.101878 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:52.102340 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:52.102840 (+ 500us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:52.105015 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:52.106386 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:52.108284 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:52.108940 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:52.109076 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:52.113327 (+ 4251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:52.113335 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:52.113348 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:52.113383 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:52.116882 (+ 3499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:52.117648 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:52.117656 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:52.117670 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:52.117831 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:52.118388 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:52.118392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:52.118394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:52.118888 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:52.119285 (+ 397us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:52.119763 (+ 478us) server_negotiation.cc:299] Negotiation successful
0809 20:22:52.120068 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":529,"threads_started":1}
I20260809 20:22:52.124967 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:22:52.124782 30545 raft_consensus.cc:348] T 00000000000000000000000000000000 P dfb2c72cb92a423f9fc52219de0dc8c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfb2c72cb92a423f9fc52219de0dc8c8" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 35367 } }
I20260809 20:22:52.125172 30545 raft_consensus.cc:374] T 00000000000000000000000000000000 P dfb2c72cb92a423f9fc52219de0dc8c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:52.125283 30545 raft_consensus.cc:729] T 00000000000000000000000000000000 P dfb2c72cb92a423f9fc52219de0dc8c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfb2c72cb92a423f9fc52219de0dc8c8, State: Initialized, Role: FOLLOWER
I20260809 20:22:52.125810 30545 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfb2c72cb92a423f9fc52219de0dc8c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfb2c72cb92a423f9fc52219de0dc8c8" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 35367 } }
I20260809 20:22:52.126066 30545 raft_consensus.cc:388] T 00000000000000000000000000000000 P dfb2c72cb92a423f9fc52219de0dc8c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:22:52.126214 30545 raft_consensus.cc:482] T 00000000000000000000000000000000 P dfb2c72cb92a423f9fc52219de0dc8c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:22:52.126451 30545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dfb2c72cb92a423f9fc52219de0dc8c8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:52.128868 30545 raft_consensus.cc:504] T 00000000000000000000000000000000 P dfb2c72cb92a423f9fc52219de0dc8c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfb2c72cb92a423f9fc52219de0dc8c8" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 35367 } }
I20260809 20:22:52.129346 30545 leader_election.cc:302] T 00000000000000000000000000000000 P dfb2c72cb92a423f9fc52219de0dc8c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dfb2c72cb92a423f9fc52219de0dc8c8; no voters:
I20260809 20:22:52.130040 30545 leader_election.cc:288] T 00000000000000000000000000000000 P dfb2c72cb92a423f9fc52219de0dc8c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:22:52.130136 30555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dfb2c72cb92a423f9fc52219de0dc8c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:52.132759 30545 sys_catalog.cc:564] T 00000000000000000000000000000000 P dfb2c72cb92a423f9fc52219de0dc8c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:22:52.137593 30555 raft_consensus.cc:686] T 00000000000000000000000000000000 P dfb2c72cb92a423f9fc52219de0dc8c8 [term 1 LEADER]: Becoming Leader. State: Replica: dfb2c72cb92a423f9fc52219de0dc8c8, State: Running, Role: LEADER
I20260809 20:22:52.138219 30555 consensus_queue.cc:237] T 00000000000000000000000000000000 P dfb2c72cb92a423f9fc52219de0dc8c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfb2c72cb92a423f9fc52219de0dc8c8" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 35367 } }
I20260809 20:22:52.142288 30558 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfb2c72cb92a423f9fc52219de0dc8c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dfb2c72cb92a423f9fc52219de0dc8c8. Latest consensus state: current_term: 1 leader_uuid: "dfb2c72cb92a423f9fc52219de0dc8c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfb2c72cb92a423f9fc52219de0dc8c8" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 35367 } } }
I20260809 20:22:52.142724 30558 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfb2c72cb92a423f9fc52219de0dc8c8 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:52.142421 30559 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfb2c72cb92a423f9fc52219de0dc8c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dfb2c72cb92a423f9fc52219de0dc8c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfb2c72cb92a423f9fc52219de0dc8c8" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 35367 } } }
I20260809 20:22:52.143683 30559 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfb2c72cb92a423f9fc52219de0dc8c8 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:52.147527 30548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:52.129698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46251 (local address 127.5.61.62:35367)
0809 20:22:52.129909 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:52.129918 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:52.130468 (+ 550us) server_negotiation.cc:407] Connection header received
0809 20:22:52.130720 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:52.130729 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:52.130821 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:52.130921 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:52.132883 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:52.133707 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:52.135469 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:52.136312 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:52.136476 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:52.139275 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:52.139280 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:52.139283 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:52.139312 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:52.144066 (+ 4754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:52.145028 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:52.145033 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:52.145036 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:52.145183 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:52.145819 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:52.145827 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:52.145831 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:52.146296 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:52.146607 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:52.146996 (+ 389us) server_negotiation.cc:299] Negotiation successful
0809 20:22:52.147173 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 20:22:52.149104 30574 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dfb2c72cb92a423f9fc52219de0dc8c8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 20:22:52.149274 30574 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 20:22:52.149427 30575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:22:52.151533 30575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:22:52.159772 30575 catalog_manager.cc:1347] Generated new cluster ID: f4170d0629a949ee8dd0d632c2a4d39b
I20260809 20:22:52.159878 30575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:22:52.178012 30575 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:22:52.179742 30575 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:22:52.199195 30575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dfb2c72cb92a423f9fc52219de0dc8c8: Generated new TSK 0
I20260809 20:22:52.200048 30575 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:22:52.207528 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 30464
I20260809 20:22:52.209113 30539 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:52.298919 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 30464
W20260809 20:22:52.319736 30447 connection.cc:535] client connection to 127.5.61.62:35367 recv error: Network error: failed to read from TLS socket (remote: 127.5.61.62:35367): Connection reset by peer (error 104)
2026-08-09T20:22:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2162 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:22:52 dist-test-slave-6k22 krb5kdc[30583](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:22:52 dist-test-slave-6k22 krb5kdc[30583](info): set up 2 sockets
Aug 09 20:22:52 dist-test-slave-6k22 krb5kdc[30583](info): commencing operation
krb5kdc: starting...
W20260809 20:22:53.503626 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:22:53 dist-test-slave-6k22 krb5kdc[30583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:22:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:22:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:22:53.813073 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:38671
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:45329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:38671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:54.141330 30597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:54.141589 30597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:54.141677 30597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:54.148607 30597 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:22:54.148703 30597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:54.148756 30597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:54.148818 30597 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:22:54.148862 30597 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:22:54.158088 30597 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:45329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:38671
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:38671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:54.159245 30597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:54.160501 30597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:54.172302 30604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:54.172209 30605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:54.172266 30607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:54.173317 30597 server_base.cc:1034] running on GCE node
I20260809 20:22:54.174535 30597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:54.176616 30597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:54.177876 30597 hybrid_clock.cc:648] HybridClock initialized: now 1786306974177807 us; error 56 us; skew 500 ppm
I20260809 20:22:54.178309 30597 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:54 dist-test-slave-6k22 krb5kdc[30583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306974, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:54.187265 30597 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:22:54.188709 30597 webserver.cc:460] Webserver started at http://127.5.61.62:44085/ using document root <none> and password file <none>
I20260809 20:22:54.189375 30597 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:54.189507 30597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:54.189817 30597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:54.192858 30597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "dcfe92ab5e7c4ddf8b06c901ca4d08a1"
format_stamp: "Formatted at 2026-08-09 20:22:54 on dist-test-slave-6k22"
server_key: "3bad8fce94f8354601754c9928dbe208"
server_key_iv: "a8ac5d4cd6c7b590b6d42f879655165b"
server_key_version: "encryptionkey@0"
I20260809 20:22:54.193579 30597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "dcfe92ab5e7c4ddf8b06c901ca4d08a1"
format_stamp: "Formatted at 2026-08-09 20:22:54 on dist-test-slave-6k22"
server_key: "3bad8fce94f8354601754c9928dbe208"
server_key_iv: "a8ac5d4cd6c7b590b6d42f879655165b"
server_key_version: "encryptionkey@0"
I20260809 20:22:54.198642 30597 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 20:22:54.201923 30614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:54.203109 30597 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 20:22:54.203317 30597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "dcfe92ab5e7c4ddf8b06c901ca4d08a1"
format_stamp: "Formatted at 2026-08-09 20:22:54 on dist-test-slave-6k22"
server_key: "3bad8fce94f8354601754c9928dbe208"
server_key_iv: "a8ac5d4cd6c7b590b6d42f879655165b"
server_key_version: "encryptionkey@0"
I20260809 20:22:54.203512 30597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:54.235236 30597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:54.239233 30597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:54.239519 30597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:54.258018 30597 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:38671
I20260809 20:22:54.258098 30682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:38671 every 8 connection(s)
I20260809 20:22:54.259353 30597 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:22:54.261284 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 30597
I20260809 20:22:54.261499 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:22:54.262048 5364 external_mini_cluster.cc:1428] Setting key 1187a5e4bed21f6c2b5f66b302f1c822
I20260809 20:22:54.263540 30683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:54.274600 30687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:54.263884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37029 (local address 127.5.61.62:38671)
0809 20:22:54.264618 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:54.264629 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:54.264665 (+ 36us) server_negotiation.cc:407] Connection header received
0809 20:22:54.266195 (+ 1530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:54.266214 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:54.266587 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:54.266991 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:54.267512 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:54.268886 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:54.270262 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:54.270915 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:54.271103 (+ 188us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:54.272462 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:54.272469 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:54.272485 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:54.273019 (+ 534us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:54.273088 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:54.273115 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:54.273293 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:54.273559 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 20:22:54.273719 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":323,"threads_started":1}
I20260809 20:22:54.274701 30683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcfe92ab5e7c4ddf8b06c901ca4d08a1: Bootstrap starting.
I20260809 20:22:54.278018 30683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcfe92ab5e7c4ddf8b06c901ca4d08a1: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:54.278344 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:22:54.279104 30683 log.cc:824] T 00000000000000000000000000000000 P dcfe92ab5e7c4ddf8b06c901ca4d08a1: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:54.281917 30683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcfe92ab5e7c4ddf8b06c901ca4d08a1: No bootstrap required, opened a new log
I20260809 20:22:54.289088 30687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:54.282743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54431 (local address 127.5.61.62:38671)
0809 20:22:54.282915 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:54.282918 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:54.283433 (+ 515us) server_negotiation.cc:407] Connection header received
0809 20:22:54.283657 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:54.283669 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:54.283741 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:54.283857 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:54.284359 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:54.285125 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:54.286472 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:54.286952 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:54.287099 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:54.288346 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:54.288351 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:54.288354 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:54.288371 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:54.288411 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:54.288426 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:54.288572 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:54.288783 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 20:22:54.288834 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 20:22:54.290330 30683 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcfe92ab5e7c4ddf8b06c901ca4d08a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcfe92ab5e7c4ddf8b06c901ca4d08a1" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38671 } }
I20260809 20:22:54.290783 30683 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcfe92ab5e7c4ddf8b06c901ca4d08a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:54.290895 30683 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcfe92ab5e7c4ddf8b06c901ca4d08a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcfe92ab5e7c4ddf8b06c901ca4d08a1, State: Initialized, Role: FOLLOWER
I20260809 20:22:54.291412 30683 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcfe92ab5e7c4ddf8b06c901ca4d08a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcfe92ab5e7c4ddf8b06c901ca4d08a1" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38671 } }
I20260809 20:22:54.291703 30683 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcfe92ab5e7c4ddf8b06c901ca4d08a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:22:54.291846 30683 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcfe92ab5e7c4ddf8b06c901ca4d08a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:22:54.292050 30683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcfe92ab5e7c4ddf8b06c901ca4d08a1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:54.294132 30683 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcfe92ab5e7c4ddf8b06c901ca4d08a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcfe92ab5e7c4ddf8b06c901ca4d08a1" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38671 } }
I20260809 20:22:54.294649 30683 leader_election.cc:302] T 00000000000000000000000000000000 P dcfe92ab5e7c4ddf8b06c901ca4d08a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcfe92ab5e7c4ddf8b06c901ca4d08a1; no voters:
I20260809 20:22:54.295212 30683 leader_election.cc:288] T 00000000000000000000000000000000 P dcfe92ab5e7c4ddf8b06c901ca4d08a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:22:54.295339 30695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcfe92ab5e7c4ddf8b06c901ca4d08a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:54.296903 30695 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcfe92ab5e7c4ddf8b06c901ca4d08a1 [term 1 LEADER]: Becoming Leader. State: Replica: dcfe92ab5e7c4ddf8b06c901ca4d08a1, State: Running, Role: LEADER
I20260809 20:22:54.297456 30695 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcfe92ab5e7c4ddf8b06c901ca4d08a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcfe92ab5e7c4ddf8b06c901ca4d08a1" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38671 } }
I20260809 20:22:54.297528 30683 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcfe92ab5e7c4ddf8b06c901ca4d08a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:22:54.300371 30697 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcfe92ab5e7c4ddf8b06c901ca4d08a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcfe92ab5e7c4ddf8b06c901ca4d08a1. Latest consensus state: current_term: 1 leader_uuid: "dcfe92ab5e7c4ddf8b06c901ca4d08a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcfe92ab5e7c4ddf8b06c901ca4d08a1" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38671 } } }
I20260809 20:22:54.300828 30697 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcfe92ab5e7c4ddf8b06c901ca4d08a1 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:54.301486 30696 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcfe92ab5e7c4ddf8b06c901ca4d08a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcfe92ab5e7c4ddf8b06c901ca4d08a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcfe92ab5e7c4ddf8b06c901ca4d08a1" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 38671 } } }
I20260809 20:22:54.301884 30696 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcfe92ab5e7c4ddf8b06c901ca4d08a1 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:54.302892 30704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:22:54.308101 30704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:22:54.318251 30704 catalog_manager.cc:1347] Generated new cluster ID: 2ec47c32ef684f7ba4d3ebc1917658de
I20260809 20:22:54.318379 30704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:22:54.338274 30704 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:22:54.339524 30704 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:22:54.356388 30704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcfe92ab5e7c4ddf8b06c901ca4d08a1: Generated new TSK 0
I20260809 20:22:54.357203 30704 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:22:54.369343 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 30597
W20260809 20:22:54.371270 30677 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37029
W20260809 20:22:54.372100 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 30597: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 20:22:54.372236 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 30597
W20260809 20:22:54.389469 30580 connection.cc:535] client connection to 127.5.61.62:38671 recv error: Network error: failed to read from TLS socket (remote: 127.5.61.62:38671): Connection reset by peer (error 104)
2026-08-09T20:22:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2069 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:22:54 dist-test-slave-6k22 krb5kdc[30717](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:22:54 dist-test-slave-6k22 krb5kdc[30717](info): set up 2 sockets
Aug 09 20:22:54 dist-test-slave-6k22 krb5kdc[30717](info): commencing operation
krb5kdc: starting...
W20260809 20:22:55.570091 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:22:55 dist-test-slave-6k22 krb5kdc[30717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:22:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:22:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:22:55.870519 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:43055
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:41547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:43055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:56.211741 30731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:56.212036 30731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:56.212136 30731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:56.219170 30731 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:22:56.219280 30731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:56.219347 30731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:56.219408 30731 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:22:56.219460 30731 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:22:56.229096 30731 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:41547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:43055
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:43055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:56.230295 30731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:56.231531 30731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:56.243937 30737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:56.243988 30738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:56.244010 30741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:56.244577 30731 server_base.cc:1034] running on GCE node
I20260809 20:22:56.245326 30731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:56.246678 30731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:56.247906 30731 hybrid_clock.cc:648] HybridClock initialized: now 1786306976247864 us; error 51 us; skew 500 ppm
I20260809 20:22:56.248238 30731 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:56 dist-test-slave-6k22 krb5kdc[30717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306976, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:56.251284 30731 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:22:56.252792 30731 webserver.cc:460] Webserver started at http://127.5.61.62:40591/ using document root <none> and password file <none>
I20260809 20:22:56.253451 30731 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:56.253625 30731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:56.253971 30731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:56.256996 30731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "e96a03189b6f4cc7949a176082bc2a12"
format_stamp: "Formatted at 2026-08-09 20:22:56 on dist-test-slave-6k22"
server_key: "e81a030df87c3ef80809336fcefd319d"
server_key_iv: "a3d008b20dd6ac9c37ed62bc56c0c8b1"
server_key_version: "encryptionkey@0"
I20260809 20:22:56.257757 30731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "e96a03189b6f4cc7949a176082bc2a12"
format_stamp: "Formatted at 2026-08-09 20:22:56 on dist-test-slave-6k22"
server_key: "e81a030df87c3ef80809336fcefd319d"
server_key_iv: "a3d008b20dd6ac9c37ed62bc56c0c8b1"
server_key_version: "encryptionkey@0"
I20260809 20:22:56.262771 30731 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 20:22:56.266135 30748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:56.267481 30731 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 20:22:56.267724 30731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "e96a03189b6f4cc7949a176082bc2a12"
format_stamp: "Formatted at 2026-08-09 20:22:56 on dist-test-slave-6k22"
server_key: "e81a030df87c3ef80809336fcefd319d"
server_key_iv: "a3d008b20dd6ac9c37ed62bc56c0c8b1"
server_key_version: "encryptionkey@0"
I20260809 20:22:56.267940 30731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:56.273481 30731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:56.277494 30731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:56.277784 30731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:56.295957 30731 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:43055
I20260809 20:22:56.296031 30806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:43055 every 8 connection(s)
I20260809 20:22:56.297324 30731 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:22:56.300887 30807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:56.307281 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 30731
I20260809 20:22:56.307514 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:22:56.307972 5364 external_mini_cluster.cc:1428] Setting key c2302927d25614d222231945e4d71bb7
I20260809 20:22:56.311008 30807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e96a03189b6f4cc7949a176082bc2a12: Bootstrap starting.
I20260809 20:22:56.314504 30807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e96a03189b6f4cc7949a176082bc2a12: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:56.315634 30812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:56.309760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58623 (local address 127.5.61.62:43055)
0809 20:22:56.310574 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:56.310584 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:56.310623 (+ 39us) server_negotiation.cc:407] Connection header received
0809 20:22:56.312323 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:56.312348 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:56.312674 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:56.312991 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:56.314094 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:56.314100 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:56.314111 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:56.314133 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:56.314186 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:56.314207 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:56.314340 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:56.314533 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 20:22:56.314726 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":363,"threads_started":1}
I20260809 20:22:56.315660 30807 log.cc:824] T 00000000000000000000000000000000 P e96a03189b6f4cc7949a176082bc2a12: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:56.318584 30807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e96a03189b6f4cc7949a176082bc2a12: No bootstrap required, opened a new log
I20260809 20:22:56.319326 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:22:56.326102 30812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:56.323486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57289 (local address 127.5.61.62:43055)
0809 20:22:56.323695 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:56.323701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:56.324073 (+ 372us) server_negotiation.cc:407] Connection header received
0809 20:22:56.324268 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:56.324277 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:56.324324 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:56.324444 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:56.325416 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:56.325422 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:56.325430 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:22:56.325448 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:56.325496 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:22:56.325514 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:56.325647 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:56.325764 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 20:22:56.325819 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 20:22:56.326633 30807 raft_consensus.cc:348] T 00000000000000000000000000000000 P e96a03189b6f4cc7949a176082bc2a12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96a03189b6f4cc7949a176082bc2a12" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 43055 } }
I20260809 20:22:56.326988 30807 raft_consensus.cc:374] T 00000000000000000000000000000000 P e96a03189b6f4cc7949a176082bc2a12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:56.327109 30807 raft_consensus.cc:729] T 00000000000000000000000000000000 P e96a03189b6f4cc7949a176082bc2a12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e96a03189b6f4cc7949a176082bc2a12, State: Initialized, Role: FOLLOWER
I20260809 20:22:56.327704 30807 consensus_queue.cc:260] T 00000000000000000000000000000000 P e96a03189b6f4cc7949a176082bc2a12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96a03189b6f4cc7949a176082bc2a12" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 43055 } }
I20260809 20:22:56.327979 30807 raft_consensus.cc:388] T 00000000000000000000000000000000 P e96a03189b6f4cc7949a176082bc2a12 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:22:56.328119 30807 raft_consensus.cc:482] T 00000000000000000000000000000000 P e96a03189b6f4cc7949a176082bc2a12 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:22:56.328308 30807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e96a03189b6f4cc7949a176082bc2a12 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:56.330381 30807 raft_consensus.cc:504] T 00000000000000000000000000000000 P e96a03189b6f4cc7949a176082bc2a12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96a03189b6f4cc7949a176082bc2a12" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 43055 } }
I20260809 20:22:56.330834 30807 leader_election.cc:302] T 00000000000000000000000000000000 P e96a03189b6f4cc7949a176082bc2a12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e96a03189b6f4cc7949a176082bc2a12; no voters:
I20260809 20:22:56.331383 30807 leader_election.cc:288] T 00000000000000000000000000000000 P e96a03189b6f4cc7949a176082bc2a12 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:22:56.331501 30820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e96a03189b6f4cc7949a176082bc2a12 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:56.332165 30820 raft_consensus.cc:686] T 00000000000000000000000000000000 P e96a03189b6f4cc7949a176082bc2a12 [term 1 LEADER]: Becoming Leader. State: Replica: e96a03189b6f4cc7949a176082bc2a12, State: Running, Role: LEADER
I20260809 20:22:56.332717 30820 consensus_queue.cc:237] T 00000000000000000000000000000000 P e96a03189b6f4cc7949a176082bc2a12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96a03189b6f4cc7949a176082bc2a12" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 43055 } }
I20260809 20:22:56.333382 30807 sys_catalog.cc:564] T 00000000000000000000000000000000 P e96a03189b6f4cc7949a176082bc2a12 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:22:56.335995 30821 sys_catalog.cc:455] T 00000000000000000000000000000000 P e96a03189b6f4cc7949a176082bc2a12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e96a03189b6f4cc7949a176082bc2a12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96a03189b6f4cc7949a176082bc2a12" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 43055 } } }
I20260809 20:22:56.336467 30821 sys_catalog.cc:458] T 00000000000000000000000000000000 P e96a03189b6f4cc7949a176082bc2a12 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:56.337596 30822 sys_catalog.cc:455] T 00000000000000000000000000000000 P e96a03189b6f4cc7949a176082bc2a12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e96a03189b6f4cc7949a176082bc2a12. Latest consensus state: current_term: 1 leader_uuid: "e96a03189b6f4cc7949a176082bc2a12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96a03189b6f4cc7949a176082bc2a12" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 43055 } } }
I20260809 20:22:56.338008 30822 sys_catalog.cc:458] T 00000000000000000000000000000000 P e96a03189b6f4cc7949a176082bc2a12 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:56.338493 30831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:22:56.344805 30831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:22:56.354610 30831 catalog_manager.cc:1347] Generated new cluster ID: 3bef42f217454e989a3df163b7019e63
I20260809 20:22:56.354709 30831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:22:56.381075 30831 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:22:56.381601 30831 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:22:56.393894 30831 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e96a03189b6f4cc7949a176082bc2a12: Generated new TSK 0
I20260809 20:22:56.394963 30831 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:22:56.399474 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 30731
W20260809 20:22:56.401198 30802 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58623
W20260809 20:22:56.401815 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 30731: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 20:22:56.401976 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 30731
2026-08-09T20:22:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2034 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:22:56 dist-test-slave-6k22 krb5kdc[30845](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:22:56 dist-test-slave-6k22 krb5kdc[30845](info): set up 2 sockets
Aug 09 20:22:56 dist-test-slave-6k22 krb5kdc[30845](info): commencing operation
krb5kdc: starting...
W20260809 20:22:57.595419 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:22:57 dist-test-slave-6k22 krb5kdc[30845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:22:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:22:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:22:57.894724 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:39171
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:46723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:39171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:22:58.223129 30859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:22:58.223448 30859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:22:58.223613 30859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:22:58.230567 30859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:22:58.230717 30859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:22:58.230822 30859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:22:58.230917 30859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:22:58.231001 30859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:22:58.240331 30859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:46723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:39171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:39171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:22:58.241509 30859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:22:58.242766 30859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:22:58.254405 30864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:58.254455 30868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:22:58.254479 30865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:22:58.255399 30859 server_base.cc:1034] running on GCE node
I20260809 20:22:58.256119 30859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:22:58.257445 30859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:22:58.258646 30859 hybrid_clock.cc:648] HybridClock initialized: now 1786306978258600 us; error 41 us; skew 500 ppm
I20260809 20:22:58.258976 30859 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:22:58 dist-test-slave-6k22 krb5kdc[30845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306978, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:22:58.262262 30859 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:22:58.263736 30859 webserver.cc:460] Webserver started at http://127.5.61.62:46691/ using document root <none> and password file <none>
I20260809 20:22:58.264403 30859 fs_manager.cc:359] Metadata directory not provided
I20260809 20:22:58.264529 30859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:22:58.264840 30859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:22:58.267757 30859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "79fd4de169a144f7ac9df9e4d0e26e86"
format_stamp: "Formatted at 2026-08-09 20:22:58 on dist-test-slave-6k22"
server_key: "2f786c983bd48557d8c3adf85cc564e6"
server_key_iv: "93b68a643737299766924bd2daa93758"
server_key_version: "encryptionkey@0"
I20260809 20:22:58.268502 30859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "79fd4de169a144f7ac9df9e4d0e26e86"
format_stamp: "Formatted at 2026-08-09 20:22:58 on dist-test-slave-6k22"
server_key: "2f786c983bd48557d8c3adf85cc564e6"
server_key_iv: "93b68a643737299766924bd2daa93758"
server_key_version: "encryptionkey@0"
I20260809 20:22:58.273437 30859 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 20:22:58.276723 30875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:22:58.277993 30859 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 20:22:58.278173 30859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "79fd4de169a144f7ac9df9e4d0e26e86"
format_stamp: "Formatted at 2026-08-09 20:22:58 on dist-test-slave-6k22"
server_key: "2f786c983bd48557d8c3adf85cc564e6"
server_key_iv: "93b68a643737299766924bd2daa93758"
server_key_version: "encryptionkey@0"
I20260809 20:22:58.278360 30859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:22:58.298992 30859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:22:58.302817 30859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:22:58.303088 30859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:22:58.321324 30859 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:39171
I20260809 20:22:58.321374 30933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:39171 every 8 connection(s)
I20260809 20:22:58.322623 30859 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:22:58.326180 30934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:22:58.331353 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 30859
I20260809 20:22:58.331554 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:22:58.332094 5364 external_mini_cluster.cc:1428] Setting key 055246b211feaf7df2e987d276ef4ecc
I20260809 20:22:58.337224 30934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79fd4de169a144f7ac9df9e4d0e26e86: Bootstrap starting.
I20260809 20:22:58.341171 30934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79fd4de169a144f7ac9df9e4d0e26e86: Neither blocks nor log segments found. Creating new log.
I20260809 20:22:58.342381 30934 log.cc:824] T 00000000000000000000000000000000 P 79fd4de169a144f7ac9df9e4d0e26e86: Log is configured to *not* fsync() on all Append() calls
I20260809 20:22:58.345856 30934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79fd4de169a144f7ac9df9e4d0e26e86: No bootstrap required, opened a new log
Aug 09 20:22:58 dist-test-slave-6k22 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:22:58.353839 30934 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79fd4de169a144f7ac9df9e4d0e26e86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79fd4de169a144f7ac9df9e4d0e26e86" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39171 } }
I20260809 20:22:58.354234 30934 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79fd4de169a144f7ac9df9e4d0e26e86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:22:58.354372 30934 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79fd4de169a144f7ac9df9e4d0e26e86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79fd4de169a144f7ac9df9e4d0e26e86, State: Initialized, Role: FOLLOWER
I20260809 20:22:58.355126 30934 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79fd4de169a144f7ac9df9e4d0e26e86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79fd4de169a144f7ac9df9e4d0e26e86" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39171 } }
I20260809 20:22:58.355430 30934 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79fd4de169a144f7ac9df9e4d0e26e86 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:22:58.355609 30934 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79fd4de169a144f7ac9df9e4d0e26e86 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:22:58.355810 30934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79fd4de169a144f7ac9df9e4d0e26e86 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:22:58.356070 30937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:58.333857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58203 (local address 127.5.61.62:39171)
0809 20:22:58.334636 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:58.334661 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:58.334708 (+ 47us) server_negotiation.cc:407] Connection header received
0809 20:22:58.336455 (+ 1747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:58.336476 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:58.336858 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:58.337154 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:58.339051 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:58.340558 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:58.342458 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:58.343546 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:58.343787 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:58.348657 (+ 4870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:58.348664 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:58.348678 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:58.348736 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:58.351923 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:58.352616 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:58.352633 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:58.352639 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:58.352806 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:58.353200 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:58.353205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:58.353211 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:58.353695 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:58.354017 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:58.354467 (+ 450us) server_negotiation.cc:299] Negotiation successful
0809 20:22:58.354860 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":349,"threads_started":1}
I20260809 20:22:58.358285 30934 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79fd4de169a144f7ac9df9e4d0e26e86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79fd4de169a144f7ac9df9e4d0e26e86" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39171 } }
I20260809 20:22:58.358867 30934 leader_election.cc:302] T 00000000000000000000000000000000 P 79fd4de169a144f7ac9df9e4d0e26e86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79fd4de169a144f7ac9df9e4d0e26e86; no voters:
I20260809 20:22:58.359593 30934 leader_election.cc:288] T 00000000000000000000000000000000 P 79fd4de169a144f7ac9df9e4d0e26e86 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:22:58.359745 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:22:58.359719 30940 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79fd4de169a144f7ac9df9e4d0e26e86 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:22:58.362095 30934 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79fd4de169a144f7ac9df9e4d0e26e86 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:22:58.364605 30940 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79fd4de169a144f7ac9df9e4d0e26e86 [term 1 LEADER]: Becoming Leader. State: Replica: 79fd4de169a144f7ac9df9e4d0e26e86, State: Running, Role: LEADER
I20260809 20:22:58.365338 30940 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79fd4de169a144f7ac9df9e4d0e26e86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79fd4de169a144f7ac9df9e4d0e26e86" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39171 } }
I20260809 20:22:58.368634 30941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79fd4de169a144f7ac9df9e4d0e26e86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79fd4de169a144f7ac9df9e4d0e26e86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79fd4de169a144f7ac9df9e4d0e26e86" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39171 } } }
I20260809 20:22:58.369091 30941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79fd4de169a144f7ac9df9e4d0e26e86 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:58.369741 30942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79fd4de169a144f7ac9df9e4d0e26e86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79fd4de169a144f7ac9df9e4d0e26e86. Latest consensus state: current_term: 1 leader_uuid: "79fd4de169a144f7ac9df9e4d0e26e86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79fd4de169a144f7ac9df9e4d0e26e86" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 39171 } } }
I20260809 20:22:58.370185 30942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79fd4de169a144f7ac9df9e4d0e26e86 [sys.catalog]: This master's current role is: LEADER
I20260809 20:22:58.376708 30956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:22:58.383139 30937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:22:58.364563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35489 (local address 127.5.61.62:39171)
0809 20:22:58.364899 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0809 20:22:58.364904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:22:58.364955 (+ 51us) server_negotiation.cc:407] Connection header received
0809 20:22:58.365163 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:22:58.365168 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:22:58.365237 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:22:58.365337 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:22:58.366860 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:58.368058 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:58.370148 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:22:58.370858 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:22:58.371838 (+ 980us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:22:58.374881 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:22:58.374890 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:22:58.374897 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:22:58.374945 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:22:58.379022 (+ 4077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:58.379932 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:58.379938 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:58.379942 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:58.380068 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:22:58.380417 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:22:58.380425 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:22:58.380431 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:22:58.380917 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:22:58.381160 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 20:22:58.382532 (+ 1372us) server_negotiation.cc:299] Negotiation successful
0809 20:22:58.382765 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244}
I20260809 20:22:58.385334 30956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:22:58.394091 30956 catalog_manager.cc:1347] Generated new cluster ID: 329949a848a54882843d4403eced485c
I20260809 20:22:58.394209 30956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:22:58.404119 30956 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:22:58.405989 30956 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:22:58.412402 30956 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79fd4de169a144f7ac9df9e4d0e26e86: Generated new TSK 0
I20260809 20:22:58.413161 30956 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:22:58.418298 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 30859
I20260809 20:22:58.419540 30928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:22:58.506575 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 30859
2026-08-09T20:22:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2101 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:22:58 dist-test-slave-6k22 krb5kdc[30975](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:22:58 dist-test-slave-6k22 krb5kdc[30975](info): set up 2 sockets
Aug 09 20:22:58 dist-test-slave-6k22 krb5kdc[30975](info): commencing operation
krb5kdc: starting...
W20260809 20:22:59.734968 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:22:59 dist-test-slave-6k22 krb5kdc[30975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:22:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:22:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:23:00.034310 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:40565
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:40257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:40565
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:23:00.368463 30994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:23:00.368810 30994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:23:00.368960 30994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:23:00.376785 30994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:23:00.376924 30994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:23:00.377007 30994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:23:00.377092 30994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:23:00.377157 30994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:23:00.386518 30994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:40257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:40565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:40565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:23:00.387877 30994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:23:00.389185 30994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:23:00.401124 31000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:23:00.401081 31001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:23:00.401154 31003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:23:00.401894 30994 server_base.cc:1034] running on GCE node
I20260809 20:23:00.402577 30994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:23:00.403870 30994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:23:00.405066 30994 hybrid_clock.cc:648] HybridClock initialized: now 1786306980405007 us; error 56 us; skew 500 ppm
I20260809 20:23:00.405356 30994 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:23:00 dist-test-slave-6k22 krb5kdc[30975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306980, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:23:00.413094 30994 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:23:00.414557 30994 webserver.cc:460] Webserver started at http://127.5.61.62:40821/ using document root <none> and password file <none>
I20260809 20:23:00.415257 30994 fs_manager.cc:359] Metadata directory not provided
I20260809 20:23:00.415410 30994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:23:00.415778 30994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:23:00.418802 30994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "7c7e38dd60aa4ee7826ce05ed5e1c2a7"
format_stamp: "Formatted at 2026-08-09 20:23:00 on dist-test-slave-6k22"
server_key: "db9acfba02d5666fe4dd0539d5c5f0d4"
server_key_iv: "7db5e02e8908db165e47ff6cad87be54"
server_key_version: "encryptionkey@0"
I20260809 20:23:00.419704 30994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "7c7e38dd60aa4ee7826ce05ed5e1c2a7"
format_stamp: "Formatted at 2026-08-09 20:23:00 on dist-test-slave-6k22"
server_key: "db9acfba02d5666fe4dd0539d5c5f0d4"
server_key_iv: "7db5e02e8908db165e47ff6cad87be54"
server_key_version: "encryptionkey@0"
I20260809 20:23:00.424891 30994 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 20:23:00.428473 31014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:23:00.429787 30994 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 20:23:00.429986 30994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "7c7e38dd60aa4ee7826ce05ed5e1c2a7"
format_stamp: "Formatted at 2026-08-09 20:23:00 on dist-test-slave-6k22"
server_key: "db9acfba02d5666fe4dd0539d5c5f0d4"
server_key_iv: "7db5e02e8908db165e47ff6cad87be54"
server_key_version: "encryptionkey@0"
I20260809 20:23:00.430181 30994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:23:00.448792 30994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:23:00.453112 30994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:23:00.453404 30994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:23:00.471632 30994 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:40565
I20260809 20:23:00.471717 31074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:40565 every 8 connection(s)
I20260809 20:23:00.473006 30994 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:23:00.474421 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 30994
I20260809 20:23:00.474610 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:23:00.475159 5364 external_mini_cluster.cc:1428] Setting key f1b0e59028ff4c45cef72f13ffefdafe
I20260809 20:23:00.477571 31075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:23:00.488471 31078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:00.477009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57547 (local address 127.5.61.62:40565)
0809 20:23:00.477795 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:00.477806 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:00.477850 (+ 44us) server_negotiation.cc:407] Connection header received
0809 20:23:00.479454 (+ 1604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:00.479474 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:00.479930 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:00.480265 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:00.480890 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:00.482262 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:00.484080 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:00.484691 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:00.484827 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:00.486130 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:00.486139 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:00.486151 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:23:00.486697 (+ 546us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:00.486777 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:23:00.486801 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:00.486911 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:00.487134 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 20:23:00.487409 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":352,"threads_started":1}
I20260809 20:23:00.490013 31075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c7e38dd60aa4ee7826ce05ed5e1c2a7: Bootstrap starting.
I20260809 20:23:00.492244 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:23:00.493330 31075 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c7e38dd60aa4ee7826ce05ed5e1c2a7: Neither blocks nor log segments found. Creating new log.
I20260809 20:23:00.494336 31075 log.cc:824] T 00000000000000000000000000000000 P 7c7e38dd60aa4ee7826ce05ed5e1c2a7: Log is configured to *not* fsync() on all Append() calls
I20260809 20:23:00.497090 31075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c7e38dd60aa4ee7826ce05ed5e1c2a7: No bootstrap required, opened a new log
I20260809 20:23:00.504616 31078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:00.497410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54355 (local address 127.5.61.62:40565)
0809 20:23:00.497649 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:00.497655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:00.498058 (+ 403us) server_negotiation.cc:407] Connection header received
0809 20:23:00.498234 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:00.498237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:00.498292 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:00.498383 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:00.499010 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:00.499809 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:00.501743 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:00.502410 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:00.502574 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:00.503881 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:00.503886 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:00.503889 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:23:00.503907 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:00.503946 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:23:00.503960 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:00.504038 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:00.504184 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 20:23:00.504250 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 20:23:00.506930 31075 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c7e38dd60aa4ee7826ce05ed5e1c2a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c7e38dd60aa4ee7826ce05ed5e1c2a7" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40565 } }
I20260809 20:23:00.507306 31075 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c7e38dd60aa4ee7826ce05ed5e1c2a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:23:00.507441 31075 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c7e38dd60aa4ee7826ce05ed5e1c2a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c7e38dd60aa4ee7826ce05ed5e1c2a7, State: Initialized, Role: FOLLOWER
I20260809 20:23:00.508098 31075 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c7e38dd60aa4ee7826ce05ed5e1c2a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c7e38dd60aa4ee7826ce05ed5e1c2a7" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40565 } }
I20260809 20:23:00.508308 31075 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c7e38dd60aa4ee7826ce05ed5e1c2a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:23:00.508405 31075 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c7e38dd60aa4ee7826ce05ed5e1c2a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:23:00.508528 31075 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c7e38dd60aa4ee7826ce05ed5e1c2a7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:23:00.510664 31075 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c7e38dd60aa4ee7826ce05ed5e1c2a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c7e38dd60aa4ee7826ce05ed5e1c2a7" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40565 } }
I20260809 20:23:00.511070 31075 leader_election.cc:302] T 00000000000000000000000000000000 P 7c7e38dd60aa4ee7826ce05ed5e1c2a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c7e38dd60aa4ee7826ce05ed5e1c2a7; no voters:
I20260809 20:23:00.511758 31075 leader_election.cc:288] T 00000000000000000000000000000000 P 7c7e38dd60aa4ee7826ce05ed5e1c2a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:23:00.511868 31085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c7e38dd60aa4ee7826ce05ed5e1c2a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:23:00.512560 31085 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c7e38dd60aa4ee7826ce05ed5e1c2a7 [term 1 LEADER]: Becoming Leader. State: Replica: 7c7e38dd60aa4ee7826ce05ed5e1c2a7, State: Running, Role: LEADER
I20260809 20:23:00.513150 31085 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c7e38dd60aa4ee7826ce05ed5e1c2a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c7e38dd60aa4ee7826ce05ed5e1c2a7" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40565 } }
I20260809 20:23:00.513880 31075 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c7e38dd60aa4ee7826ce05ed5e1c2a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:23:00.515667 31086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c7e38dd60aa4ee7826ce05ed5e1c2a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c7e38dd60aa4ee7826ce05ed5e1c2a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c7e38dd60aa4ee7826ce05ed5e1c2a7" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40565 } } }
I20260809 20:23:00.515774 31087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c7e38dd60aa4ee7826ce05ed5e1c2a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c7e38dd60aa4ee7826ce05ed5e1c2a7. Latest consensus state: current_term: 1 leader_uuid: "7c7e38dd60aa4ee7826ce05ed5e1c2a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c7e38dd60aa4ee7826ce05ed5e1c2a7" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 40565 } } }
I20260809 20:23:00.516095 31087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c7e38dd60aa4ee7826ce05ed5e1c2a7 [sys.catalog]: This master's current role is: LEADER
I20260809 20:23:00.516095 31086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c7e38dd60aa4ee7826ce05ed5e1c2a7 [sys.catalog]: This master's current role is: LEADER
I20260809 20:23:00.518121 31091 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:23:00.523149 31091 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:23:00.531971 31091 catalog_manager.cc:1347] Generated new cluster ID: e93d181df56d4ae4ba1d71cc7c738277
I20260809 20:23:00.532075 31091 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:23:00.545126 31091 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:23:00.546921 31091 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:23:00.560189 31091 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c7e38dd60aa4ee7826ce05ed5e1c2a7: Generated new TSK 0
I20260809 20:23:00.561149 31091 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:23:00.571766 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 30994
W20260809 20:23:00.573004 31070 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57547
W20260809 20:23:00.573714 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 30994: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 20:23:00.573871 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 30994
2026-08-09T20:23:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2067 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:23:00 dist-test-slave-6k22 krb5kdc[31110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:23:00 dist-test-slave-6k22 krb5kdc[31110](info): set up 2 sockets
Aug 09 20:23:00 dist-test-slave-6k22 krb5kdc[31110](info): commencing operation
krb5kdc: starting...
W20260809 20:23:01.774405 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:23:01 dist-test-slave-6k22 krb5kdc[31110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:23:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:23:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:23:02.070794 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:34517
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:39985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:34517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:23:02.403918 31125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:23:02.404183 31125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:23:02.404299 31125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:23:02.411187 31125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:23:02.411387 31125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:23:02.411478 31125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:23:02.411550 31125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:23:02.411655 31125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:23:02.421126 31125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:39985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:34517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:34517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:23:02.422360 31125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:23:02.423621 31125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:23:02.435389 31130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:23:02.435500 31136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:23:02.435529 31133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:23:02.436604 31125 server_base.cc:1034] running on GCE node
I20260809 20:23:02.437384 31125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:23:02.438725 31125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:23:02.439951 31125 hybrid_clock.cc:648] HybridClock initialized: now 1786306982439923 us; error 53 us; skew 500 ppm
I20260809 20:23:02.440297 31125 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:23:02 dist-test-slave-6k22 krb5kdc[31110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306982, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:23:02.443334 31125 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:23:02.444845 31125 webserver.cc:460] Webserver started at http://127.5.61.62:42623/ using document root <none> and password file <none>
I20260809 20:23:02.445501 31125 fs_manager.cc:359] Metadata directory not provided
I20260809 20:23:02.445647 31125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:23:02.445957 31125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:23:02.448923 31125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "313bf0c3e8fd459c8b20a85ed7580c54"
format_stamp: "Formatted at 2026-08-09 20:23:02 on dist-test-slave-6k22"
server_key: "eb6e44805f53a84666ffc9b5f5d471df"
server_key_iv: "b15a3e640de054987462ff5c4008b1c0"
server_key_version: "encryptionkey@0"
I20260809 20:23:02.449772 31125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "313bf0c3e8fd459c8b20a85ed7580c54"
format_stamp: "Formatted at 2026-08-09 20:23:02 on dist-test-slave-6k22"
server_key: "eb6e44805f53a84666ffc9b5f5d471df"
server_key_iv: "b15a3e640de054987462ff5c4008b1c0"
server_key_version: "encryptionkey@0"
I20260809 20:23:02.454697 31125 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 20:23:02.457970 31143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:23:02.459246 31125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 20:23:02.459427 31125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "313bf0c3e8fd459c8b20a85ed7580c54"
format_stamp: "Formatted at 2026-08-09 20:23:02 on dist-test-slave-6k22"
server_key: "eb6e44805f53a84666ffc9b5f5d471df"
server_key_iv: "b15a3e640de054987462ff5c4008b1c0"
server_key_version: "encryptionkey@0"
I20260809 20:23:02.459649 31125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:23:02.465198 31125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:23:02.468974 31125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:23:02.469271 31125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:23:02.487763 31125 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:34517
I20260809 20:23:02.487810 31205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:34517 every 8 connection(s)
I20260809 20:23:02.489121 31125 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:23:02.492770 31206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:23:02.494380 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 31125
I20260809 20:23:02.494591 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:23:02.495007 5364 external_mini_cluster.cc:1428] Setting key c1446eaa7579826c4cd5e39fdffe5bf5
I20260809 20:23:02.502974 31209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:02.496829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43819 (local address 127.5.61.62:34517)
0809 20:23:02.497628 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:02.497640 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:02.497685 (+ 45us) server_negotiation.cc:407] Connection header received
0809 20:23:02.499218 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:02.499249 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:02.499648 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:02.499980 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:02.501063 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:02.501069 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:02.501081 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:23:02.501095 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:02.501150 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:23:02.501169 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:02.501375 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:02.501537 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 20:23:02.501735 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":375,"threads_started":1}
I20260809 20:23:02.506388 31206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 313bf0c3e8fd459c8b20a85ed7580c54: Bootstrap starting.
I20260809 20:23:02.506587 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:23:02.510208 31206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 313bf0c3e8fd459c8b20a85ed7580c54: Neither blocks nor log segments found. Creating new log.
I20260809 20:23:02.511678 31206 log.cc:824] T 00000000000000000000000000000000 P 313bf0c3e8fd459c8b20a85ed7580c54: Log is configured to *not* fsync() on all Append() calls
I20260809 20:23:02.514537 31209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:02.512000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59691 (local address 127.5.61.62:34517)
0809 20:23:02.512208 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:02.512216 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:02.512603 (+ 387us) server_negotiation.cc:407] Connection header received
0809 20:23:02.512774 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:02.512777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:02.512820 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:02.512898 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:02.513913 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:02.513921 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:02.513927 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 20:23:02.513944 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:02.513987 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 20:23:02.514013 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:02.514124 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:02.514224 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 20:23:02.514281 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 20:23:02.515072 31206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 313bf0c3e8fd459c8b20a85ed7580c54: No bootstrap required, opened a new log
I20260809 20:23:02.522289 31206 raft_consensus.cc:348] T 00000000000000000000000000000000 P 313bf0c3e8fd459c8b20a85ed7580c54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "313bf0c3e8fd459c8b20a85ed7580c54" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 34517 } }
I20260809 20:23:02.522621 31206 raft_consensus.cc:374] T 00000000000000000000000000000000 P 313bf0c3e8fd459c8b20a85ed7580c54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 20:23:02.522691 31206 raft_consensus.cc:729] T 00000000000000000000000000000000 P 313bf0c3e8fd459c8b20a85ed7580c54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 313bf0c3e8fd459c8b20a85ed7580c54, State: Initialized, Role: FOLLOWER
I20260809 20:23:02.523140 31206 consensus_queue.cc:260] T 00000000000000000000000000000000 P 313bf0c3e8fd459c8b20a85ed7580c54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "313bf0c3e8fd459c8b20a85ed7580c54" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 34517 } }
I20260809 20:23:02.523370 31206 raft_consensus.cc:388] T 00000000000000000000000000000000 P 313bf0c3e8fd459c8b20a85ed7580c54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 20:23:02.523524 31206 raft_consensus.cc:482] T 00000000000000000000000000000000 P 313bf0c3e8fd459c8b20a85ed7580c54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 20:23:02.523747 31206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 313bf0c3e8fd459c8b20a85ed7580c54 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:23:02.525897 31206 raft_consensus.cc:504] T 00000000000000000000000000000000 P 313bf0c3e8fd459c8b20a85ed7580c54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "313bf0c3e8fd459c8b20a85ed7580c54" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 34517 } }
I20260809 20:23:02.526365 31206 leader_election.cc:302] T 00000000000000000000000000000000 P 313bf0c3e8fd459c8b20a85ed7580c54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 313bf0c3e8fd459c8b20a85ed7580c54; no voters:
I20260809 20:23:02.526962 31206 leader_election.cc:288] T 00000000000000000000000000000000 P 313bf0c3e8fd459c8b20a85ed7580c54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 20:23:02.527110 31216 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 313bf0c3e8fd459c8b20a85ed7580c54 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:23:02.528357 31216 raft_consensus.cc:686] T 00000000000000000000000000000000 P 313bf0c3e8fd459c8b20a85ed7580c54 [term 1 LEADER]: Becoming Leader. State: Replica: 313bf0c3e8fd459c8b20a85ed7580c54, State: Running, Role: LEADER
I20260809 20:23:02.528882 31216 consensus_queue.cc:237] T 00000000000000000000000000000000 P 313bf0c3e8fd459c8b20a85ed7580c54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "313bf0c3e8fd459c8b20a85ed7580c54" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 34517 } }
I20260809 20:23:02.529094 31206 sys_catalog.cc:564] T 00000000000000000000000000000000 P 313bf0c3e8fd459c8b20a85ed7580c54 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:23:02.531731 31217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 313bf0c3e8fd459c8b20a85ed7580c54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "313bf0c3e8fd459c8b20a85ed7580c54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "313bf0c3e8fd459c8b20a85ed7580c54" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 34517 } } }
I20260809 20:23:02.532171 31217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 313bf0c3e8fd459c8b20a85ed7580c54 [sys.catalog]: This master's current role is: LEADER
I20260809 20:23:02.532642 31219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 313bf0c3e8fd459c8b20a85ed7580c54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 313bf0c3e8fd459c8b20a85ed7580c54. Latest consensus state: current_term: 1 leader_uuid: "313bf0c3e8fd459c8b20a85ed7580c54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "313bf0c3e8fd459c8b20a85ed7580c54" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 34517 } } }
I20260809 20:23:02.533020 31219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 313bf0c3e8fd459c8b20a85ed7580c54 [sys.catalog]: This master's current role is: LEADER
I20260809 20:23:02.534585 31227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:23:02.538849 31227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:23:02.546909 31227 catalog_manager.cc:1347] Generated new cluster ID: f83f10f83ea64b79a8f8863aa542367e
I20260809 20:23:02.547019 31227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:23:02.558092 31227 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:23:02.558583 31227 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:23:02.575322 31227 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 313bf0c3e8fd459c8b20a85ed7580c54: Generated new TSK 0
I20260809 20:23:02.576413 31227 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:23:02.590286 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 31125
W20260809 20:23:02.591444 31201 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43819
W20260809 20:23:02.592260 5364 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 31125: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 20:23:02.592427 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 31125
2026-08-09T20:23:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2021 ms)
[----------] 18 tests from AuthTokenIssuingTest (37326 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:23:02 dist-test-slave-6k22 krb5kdc[31239](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:23:02 dist-test-slave-6k22 krb5kdc[31239](info): set up 2 sockets
Aug 09 20:23:02 dist-test-slave-6k22 krb5kdc[31239](info): commencing operation
krb5kdc: starting...
W20260809 20:23:03.802260 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:23:04 dist-test-slave-6k22 krb5kdc[31239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:23:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:23:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:23:04.111138 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:44379
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:44379,127.5.61.61:36693,127.5.61.60:42103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:23:04.446877 31256 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 20:23:04.447268 31256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:23:04.447393 31256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:23:04.447526 31256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:23:04.454664 31256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:23:04.454766 31256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:23:04.454878 31256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:23:04.454955 31256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:23:04.455039 31256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:23:04.464411 31256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:44379,127.5.61.61:36693,127.5.61.60:42103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:44379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:23:04.465619 31256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:23:04.466921 31256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:23:04.478560 31262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:23:04.478513 31265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:23:04.478600 31263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:23:04.479787 31256 server_base.cc:1034] running on GCE node
I20260809 20:23:04.480484 31256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:23:04.481765 31256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:23:04.482997 31256 hybrid_clock.cc:648] HybridClock initialized: now 1786306984482941 us; error 36 us; skew 500 ppm
I20260809 20:23:04.483341 31256 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:23:04 dist-test-slave-6k22 krb5kdc[31239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306984, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:23:04.486467 31256 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:23:04.488037 31256 webserver.cc:460] Webserver started at http://127.5.61.62:35083/ using document root <none> and password file <none>
I20260809 20:23:04.488693 31256 fs_manager.cc:359] Metadata directory not provided
I20260809 20:23:04.488818 31256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:23:04.489126 31256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:23:04.491995 31256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "f89bd7bd61264fc58ed9b63fbf50c042"
format_stamp: "Formatted at 2026-08-09 20:23:04 on dist-test-slave-6k22"
server_key: "e5a3064a6edebbea565244271d17153c"
server_key_iv: "4680f5d8d7f1fd3610b4e85a562d4d3a"
server_key_version: "encryptionkey@0"
I20260809 20:23:04.492739 31256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "f89bd7bd61264fc58ed9b63fbf50c042"
format_stamp: "Formatted at 2026-08-09 20:23:04 on dist-test-slave-6k22"
server_key: "e5a3064a6edebbea565244271d17153c"
server_key_iv: "4680f5d8d7f1fd3610b4e85a562d4d3a"
server_key_version: "encryptionkey@0"
I20260809 20:23:04.497478 31256 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 20:23:04.500726 31273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:23:04.501888 31256 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260809 20:23:04.502050 31256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "f89bd7bd61264fc58ed9b63fbf50c042"
format_stamp: "Formatted at 2026-08-09 20:23:04 on dist-test-slave-6k22"
server_key: "e5a3064a6edebbea565244271d17153c"
server_key_iv: "4680f5d8d7f1fd3610b4e85a562d4d3a"
server_key_version: "encryptionkey@0"
I20260809 20:23:04.502238 31256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:23:04.526005 31256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:23:04.530179 31256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:23:04.530450 31256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:23:04.548849 31256 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:44379
I20260809 20:23:04.548872 31334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:44379 every 8 connection(s)
I20260809 20:23:04.550127 31256 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:23:04.550555 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 31256
I20260809 20:23:04.550760 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:23:04.551607 5364 external_mini_cluster.cc:1428] Setting key cf892c6044f491c07c786e0d373d3f16
I20260809 20:23:04.553791 31336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:23:04.560289 31336 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44379 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 20:23:04 dist-test-slave-6k22 krb5kdc[31239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306984, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.61.61@KRBTEST.COM; defaulting to no policy
I20260809 20:23:04.582194 31338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:04.563275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51506 (local address 127.5.61.62:44379)
0809 20:23:04.564144 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:04.564162 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:04.564786 (+ 624us) server_negotiation.cc:407] Connection header received
0809 20:23:04.566747 (+ 1961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:04.566753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:04.566967 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:04.567160 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:04.568488 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:04.569631 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:04.570992 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:04.571550 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:04.571801 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:04.574972 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:04.574979 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:04.574994 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:04.575020 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:04.577861 (+ 2841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:04.578529 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:04.578538 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:04.578541 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:04.578682 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:04.579076 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:04.579083 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:04.579087 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:04.579607 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:04.579815 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:04.580316 (+ 501us) server_negotiation.cc:299] Negotiation successful
0809 20:23:04.580691 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":615,"thread_start_us":366,"threads_started":1}
I20260809 20:23:04.582212 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:04.563275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:44379 (local address 127.0.0.1:51506)
0809 20:23:04.564616 (+ 1341us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:04.564657 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 20:23:04.566394 (+ 1737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:23:04.567183 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:23:04.567189 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:23:04.567936 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:23:04.568263 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:04.568280 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:04.569788 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:04.569793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:04.570819 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:04.570831 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:04.571824 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:04.571831 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:04.572008 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:04.572671 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:23:04.572692 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:23:04.574761 (+ 2069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:23:04.577989 (+ 3228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:04.578001 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:04.578012 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:04.578403 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:04.578792 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:04.578798 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:04.578801 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:04.578971 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:04.579731 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:23:04.579751 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:23:04.579960 (+ 209us) client_negotiation.cc:769] Sending connection context
0809 20:23:04.580193 (+ 233us) client_negotiation.cc:240] Negotiation successful
0809 20:23:04.580822 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1138,"mutex_wait_us":182,"spinlock_wait_cycles":14592,"thread_start_us":952,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.61@KRBTEST.COM" created.
I20260809 20:23:04.585475 31336 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } has no permanent_uuid. Determining permanent_uuid...
I20260809 20:23:04.586618 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:04.586105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:36693 (local address 127.0.0.1:58690)
0809 20:23:04.586315 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:04.586452 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:36693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260809 20:23:04.586872 31274 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.61.61:36693: connect: Connection refused (error 111)
I20260809 20:23:04.588521 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:04.588082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:36693 (local address 127.0.0.1:58704)
0809 20:23:04.588257 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:04.588358 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:36693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 20:23:04.588744 31336 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.61:36693: Network error: Client connection negotiation failed: client connection to 127.5.61.61:36693: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.61.keytab.
Entry for principal kudu/127.5.61.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.61.keytab.
I20260809 20:23:04.622354 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.61:36693
--webserver_interface=127.5.61.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.61.keytab
--principal=kudu/127.5.61.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:44379,127.5.61.61:36693,127.5.61.60:42103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 20:23:04.638167 31336 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } attempt: 1
I20260809 20:23:04.639405 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:04.638975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:36693 (local address 127.0.0.1:58706)
0809 20:23:04.639167 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:04.639279 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:36693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 20:23:04.640451 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:04.640098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:36693 (local address 127.0.0.1:58714)
0809 20:23:04.640263 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:04.640329 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:36693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 20:23:04.640610 31336 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.61:36693: Network error: Client connection negotiation failed: client connection to 127.5.61.61:36693: connect: Connection refused (error 111)
I20260809 20:23:04.708827 31336 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } attempt: 2
I20260809 20:23:04.710107 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:04.709496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:36693 (local address 127.0.0.1:58716)
0809 20:23:04.709784 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:04.709920 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:36693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 20:23:04.711285 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:04.710848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:36693 (local address 127.0.0.1:58730)
0809 20:23:04.711006 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:04.711111 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:36693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 20:23:04.711541 31336 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.61:36693: Network error: Client connection negotiation failed: client connection to 127.5.61.61:36693: connect: Connection refused (error 111)
I20260809 20:23:04.802848 31336 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } attempt: 3
I20260809 20:23:04.804229 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:04.803521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:36693 (local address 127.0.0.1:58736)
0809 20:23:04.803837 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:04.803988 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:36693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260809 20:23:04.805544 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:04.804964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:36693 (local address 127.0.0.1:58750)
0809 20:23:04.805192 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:04.805313 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:36693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260809 20:23:04.805830 31336 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.61:36693: Network error: Client connection negotiation failed: client connection to 127.5.61.61:36693: connect: Connection refused (error 111)
I20260809 20:23:04.949119 31336 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } attempt: 4
I20260809 20:23:04.950460 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:04.949822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:36693 (local address 127.0.0.1:58766)
0809 20:23:04.950173 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:04.950284 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:36693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
I20260809 20:23:04.951601 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:04.951152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:36693 (local address 127.0.0.1:58772)
0809 20:23:04.951316 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:04.951418 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:36693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 20:23:04.951782 31336 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.61:36693: Network error: Client connection negotiation failed: client connection to 127.5.61.61:36693: connect: Connection refused (error 111)
W20260809 20:23:04.973214 31345 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 20:23:04.973577 31345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:23:04.973706 31345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:23:04.973839 31345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:23:04.981252 31345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:23:04.981341 31345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:23:04.981483 31345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:23:04.981572 31345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:23:04.981674 31345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:23:04.991422 31345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:44379,127.5.61.61:36693,127.5.61.60:42103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.61.keytab
--principal=kudu/127.5.61.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.61:36693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.61.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:23:04.992748 31345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:23:04.994141 31345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:23:05.006021 31350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:23:05.006067 31352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:23:05.006027 31354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:23:05.006986 31345 server_base.cc:1034] running on GCE node
I20260809 20:23:05.007900 31345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:23:05.009256 31345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:23:05.010514 31345 hybrid_clock.cc:648] HybridClock initialized: now 1786306985010436 us; error 53 us; skew 500 ppm
I20260809 20:23:05.010864 31345 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:23:05 dist-test-slave-6k22 krb5kdc[31239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306985, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:23:05.014452 31345 init.cc:377] Logged in from keytab as kudu/127.5.61.61@KRBTEST.COM (short username kudu)
I20260809 20:23:05.016101 31345 webserver.cc:460] Webserver started at http://127.5.61.61:40883/ using document root <none> and password file <none>
I20260809 20:23:05.016803 31345 fs_manager.cc:359] Metadata directory not provided
I20260809 20:23:05.016939 31345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:23:05.017269 31345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:23:05.020396 31345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-1/data/instance:
uuid: "73053f79361949acb3008794dfa5013c"
format_stamp: "Formatted at 2026-08-09 20:23:05 on dist-test-slave-6k22"
server_key: "6322dc59f17f3f1ba27a38beed112ee1"
server_key_iv: "585a4f63bb8907ce22e3c907a0a5e40a"
server_key_version: "encryptionkey@0"
I20260809 20:23:05.021191 31345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-1/wal/instance:
uuid: "73053f79361949acb3008794dfa5013c"
format_stamp: "Formatted at 2026-08-09 20:23:05 on dist-test-slave-6k22"
server_key: "6322dc59f17f3f1ba27a38beed112ee1"
server_key_iv: "585a4f63bb8907ce22e3c907a0a5e40a"
server_key_version: "encryptionkey@0"
I20260809 20:23:05.026340 31345 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 20:23:05.029799 31361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:23:05.031208 31345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 20:23:05.031388 31345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-1/wal
uuid: "73053f79361949acb3008794dfa5013c"
format_stamp: "Formatted at 2026-08-09 20:23:05 on dist-test-slave-6k22"
server_key: "6322dc59f17f3f1ba27a38beed112ee1"
server_key_iv: "585a4f63bb8907ce22e3c907a0a5e40a"
server_key_version: "encryptionkey@0"
I20260809 20:23:05.031617 31345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:23:05.064594 31345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:23:05.069041 31345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:23:05.069335 31345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:23:05.089179 31345 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.61:36693
I20260809 20:23:05.089236 31419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.61:36693 every 8 connection(s)
I20260809 20:23:05.090545 31345 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-1/data/info.pb
I20260809 20:23:05.092108 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 31345
I20260809 20:23:05.092329 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-1/wal/instance
I20260809 20:23:05.092928 5364 external_mini_cluster.cc:1428] Setting key 4908f673db55153188501294c73b04cb
I20260809 20:23:05.095448 31420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:23:05.106201 31420 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44379 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 20:23:05 dist-test-slave-6k22 krb5kdc[31239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306985, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.61@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.61.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.60@KRBTEST.COM" created.
I20260809 20:23:05.131608 31423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.109620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51508 (local address 127.5.61.62:44379)
0809 20:23:05.110150 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:05.110157 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:05.111214 (+ 1057us) server_negotiation.cc:407] Connection header received
0809 20:23:05.114138 (+ 2924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:05.114145 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:05.114219 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:05.114324 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:05.116477 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.117294 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.119859 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.120290 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.120389 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:05.124909 (+ 4520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:05.124914 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:05.124917 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:05.124970 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:05.128245 (+ 3275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.129054 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.129064 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.129068 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.129198 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.129645 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.129650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.129654 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.129943 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:05.130115 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:05.130938 (+ 823us) server_negotiation.cc:299] Negotiation successful
0809 20:23:05.131183 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":263,"threads_started":1}
I20260809 20:23:05.132573 31424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.110194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:44379 (local address 127.0.0.1:51508)
0809 20:23:05.111020 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.111078 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 20:23:05.113834 (+ 2756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:23:05.114572 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:23:05.114585 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:23:05.115430 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:23:05.116252 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:05.116279 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.117471 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.117479 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:05.119698 (+ 2219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:05.119715 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.120563 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.120575 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:05.120903 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:05.121680 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:23:05.121714 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:23:05.124669 (+ 2955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:23:05.128388 (+ 3719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:05.128397 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:05.128413 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:05.128869 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:05.129320 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:05.129325 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:05.129329 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:05.129515 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:05.130115 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:23:05.130124 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:23:05.130512 (+ 388us) client_negotiation.cc:769] Sending connection context
0809 20:23:05.130826 (+ 314us) client_negotiation.cc:240] Negotiation successful
0809 20:23:05.131347 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":320,"threads_started":1}
I20260809 20:23:05.134523 31420 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 20:23:05 dist-test-slave-6k22 krb5kdc[31239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306985, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.61@KRBTEST.COM for kudu/127.5.61.61@KRBTEST.COM
I20260809 20:23:05.151315 31424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.135412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:36693 (local address 127.0.0.1:58788)
0809 20:23:05.135623 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.135644 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 20:23:05.135755 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:23:05.137013 (+ 1258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:23:05.137021 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:23:05.137517 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:23:05.137731 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:05.137745 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.139809 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.139817 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:05.141278 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:05.141294 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.142508 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.142516 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:05.142776 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:05.143436 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:23:05.143461 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:23:05.145754 (+ 2293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:23:05.148882 (+ 3128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:05.148887 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:05.148891 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:05.149228 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:05.149596 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:05.149603 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:05.149606 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:05.149727 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:05.150325 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:23:05.150329 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:23:05.150483 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 20:23:05.150630 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 20:23:05.150919 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":3}
I20260809 20:23:05.151405 31426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.135732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58788 (local address 127.5.61.61:36693)
0809 20:23:05.136471 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:05.136482 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:05.136513 (+ 31us) server_negotiation.cc:407] Connection header received
0809 20:23:05.136585 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:05.136595 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:05.136839 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:05.136985 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:05.137919 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.139642 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.141438 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.142356 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.142495 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:05.145967 (+ 3472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:05.145976 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:05.145987 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:05.146017 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:05.148741 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.149359 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.149369 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.149373 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.149489 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.149835 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.149843 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.149846 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.150183 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:05.150409 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:05.150783 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 20:23:05.151023 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":536,"thread_start_us":349,"threads_started":1}
I20260809 20:23:05.153769 31420 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } has no permanent_uuid. Determining permanent_uuid...
I20260809 20:23:05.154939 31424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.154523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42103 (local address 127.0.0.1:44590)
0809 20:23:05.154666 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.154814 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 20:23:05.155215 31364 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
I20260809 20:23:05.156889 31424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.156521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42103 (local address 127.0.0.1:44602)
0809 20:23:05.156712 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.156795 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 20:23:05.157119 31420 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:42103: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.60.keytab.
Entry for principal kudu/127.5.61.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.60.keytab.
I20260809 20:23:05.171411 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.60:42103
--webserver_interface=127.5.61.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.60.keytab
--principal=kudu/127.5.61.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:44379,127.5.61.61:36693,127.5.61.60:42103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 20:23:05.206416 31420 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } attempt: 1
I20260809 20:23:05.207621 31424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.207140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42103 (local address 127.0.0.1:44616)
0809 20:23:05.207330 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.207419 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 20:23:05.208627 31424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.208324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42103 (local address 127.0.0.1:44624)
0809 20:23:05.208462 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.208528 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 20:23:05.208904 31420 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:42103: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
I20260809 20:23:05.251066 31336 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } attempt: 5
Aug 09 20:23:05 dist-test-slave-6k22 krb5kdc[31239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306984, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for kudu/127.5.61.61@KRBTEST.COM
I20260809 20:23:05.264784 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.251950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:36693 (local address 127.0.0.1:58790)
0809 20:23:05.252203 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.252227 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 20:23:05.252370 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:23:05.252776 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:23:05.252781 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:23:05.253224 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:23:05.253419 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:05.253428 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.254553 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.254559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:05.255587 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:05.255604 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.256402 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.256408 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:05.256595 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:05.257301 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:23:05.257319 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:23:05.259424 (+ 2105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:23:05.262329 (+ 2905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:05.262335 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:05.262339 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:05.262701 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:05.263088 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:05.263093 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:05.263096 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:05.263233 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:05.263815 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:23:05.263819 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:23:05.263971 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 20:23:05.264113 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 20:23:05.264433 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260809 20:23:05.264778 31426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.252061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58790 (local address 127.5.61.61:36693)
0809 20:23:05.252268 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:05.252276 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:05.252302 (+ 26us) server_negotiation.cc:407] Connection header received
0809 20:23:05.252547 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:05.252552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:05.252627 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:05.252741 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:05.253578 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.254375 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.255748 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.256215 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.256387 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:05.259647 (+ 3260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:05.259651 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:05.259655 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:05.259687 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:05.262196 (+ 2509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.262827 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.262832 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.262835 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.262971 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.263343 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.263349 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.263353 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.263698 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:05.263852 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:05.264238 (+ 386us) server_negotiation.cc:299] Negotiation successful
0809 20:23:05.264446 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 20:23:05.266108 31336 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } has no permanent_uuid. Determining permanent_uuid...
I20260809 20:23:05.267259 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.266868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42103 (local address 127.0.0.1:44628)
0809 20:23:05.267032 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.267115 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 20:23:05.268263 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.267954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42103 (local address 127.0.0.1:44636)
0809 20:23:05.268087 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.268173 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 20:23:05.268491 31336 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:42103: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
I20260809 20:23:05.277125 31420 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } attempt: 2
I20260809 20:23:05.278237 31424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.277797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42103 (local address 127.0.0.1:44642)
0809 20:23:05.277995 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.278082 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 20:23:05.279330 31424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.278961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42103 (local address 127.0.0.1:44646)
0809 20:23:05.279123 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.279203 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 20:23:05.279547 31420 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:42103: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
I20260809 20:23:05.319711 31336 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } attempt: 1
I20260809 20:23:05.320986 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.320380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42103 (local address 127.0.0.1:44662)
0809 20:23:05.320636 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.320776 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260809 20:23:05.322145 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.321709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42103 (local address 127.0.0.1:44664)
0809 20:23:05.321870 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.321956 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 20:23:05.322382 31336 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:42103: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
I20260809 20:23:05.370922 31420 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } attempt: 3
I20260809 20:23:05.372229 31424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.371747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42103 (local address 127.0.0.1:44674)
0809 20:23:05.371982 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.372071 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 20:23:05.373299 31424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.372901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42103 (local address 127.0.0.1:44688)
0809 20:23:05.373112 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.373187 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260809 20:23:05.373540 31420 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:42103: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
I20260809 20:23:05.390646 31336 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } attempt: 2
I20260809 20:23:05.392091 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.391407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42103 (local address 127.0.0.1:44702)
0809 20:23:05.391720 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.391841 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260809 20:23:05.393235 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.392795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42103 (local address 127.0.0.1:44704)
0809 20:23:05.392952 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.393055 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 20:23:05.393553 31336 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:42103: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
I20260809 20:23:05.499811 31336 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } attempt: 3
I20260809 20:23:05.501223 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.500571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42103 (local address 127.0.0.1:44706)
0809 20:23:05.500835 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.500983 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260809 20:23:05.502379 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.501923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42103 (local address 127.0.0.1:44716)
0809 20:23:05.502143 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.502241 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260809 20:23:05.502599 31336 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:42103: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
I20260809 20:23:05.516808 31420 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } attempt: 4
I20260809 20:23:05.517807 31424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.517371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42103 (local address 127.0.0.1:44726)
0809 20:23:05.517585 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.517661 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 20:23:05.518795 31424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.518474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42103 (local address 127.0.0.1:44738)
0809 20:23:05.518614 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.518687 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 20:23:05.519006 31420 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:42103: Network error: Client connection negotiation failed: client connection to 127.5.61.60:42103: connect: Connection refused (error 111)
W20260809 20:23:05.535948 31430 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 20:23:05.536324 31430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:23:05.536448 31430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:23:05.536581 31430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:23:05.543694 31430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:23:05.543799 31430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:23:05.543896 31430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:23:05.543972 31430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:23:05.544056 31430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:23:05.553546 31430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:44379,127.5.61.61:36693,127.5.61.60:42103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.60.keytab
--principal=kudu/127.5.61.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.60:42103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.61.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:23:05.554740 31430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:23:05.556061 31430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:23:05.567819 31436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:23:05.567780 31437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:23:05.567842 31439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:23:05.568814 31430 server_base.cc:1034] running on GCE node
I20260809 20:23:05.569486 31430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:23:05.570711 31430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:23:05.571969 31430 hybrid_clock.cc:648] HybridClock initialized: now 1786306985571894 us; error 53 us; skew 500 ppm
I20260809 20:23:05.572289 31430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:23:05 dist-test-slave-6k22 krb5kdc[31239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306985, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:23:05.580042 31430 init.cc:377] Logged in from keytab as kudu/127.5.61.60@KRBTEST.COM (short username kudu)
I20260809 20:23:05.581576 31430 webserver.cc:460] Webserver started at http://127.5.61.60:46205/ using document root <none> and password file <none>
I20260809 20:23:05.582273 31430 fs_manager.cc:359] Metadata directory not provided
I20260809 20:23:05.582408 31430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:23:05.582743 31430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:23:05.585845 31430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-2/data/instance:
uuid: "18670dfab7c04cdca51d63a35a2a1dff"
format_stamp: "Formatted at 2026-08-09 20:23:05 on dist-test-slave-6k22"
server_key: "822078fae068dde0bd6cfd28462a65ff"
server_key_iv: "5d834372e832eb3c95d9f77f4c64a570"
server_key_version: "encryptionkey@0"
I20260809 20:23:05.586611 31430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-2/wal/instance:
uuid: "18670dfab7c04cdca51d63a35a2a1dff"
format_stamp: "Formatted at 2026-08-09 20:23:05 on dist-test-slave-6k22"
server_key: "822078fae068dde0bd6cfd28462a65ff"
server_key_iv: "5d834372e832eb3c95d9f77f4c64a570"
server_key_version: "encryptionkey@0"
I20260809 20:23:05.591991 31430 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 20:23:05.595286 31448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:23:05.596644 31430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:23:05.596855 31430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-2/wal
uuid: "18670dfab7c04cdca51d63a35a2a1dff"
format_stamp: "Formatted at 2026-08-09 20:23:05 on dist-test-slave-6k22"
server_key: "822078fae068dde0bd6cfd28462a65ff"
server_key_iv: "5d834372e832eb3c95d9f77f4c64a570"
server_key_version: "encryptionkey@0"
I20260809 20:23:05.597077 31430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:23:05.608913 31430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:23:05.612895 31430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:23:05.613155 31430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:23:05.632151 31430 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.60:42103
I20260809 20:23:05.632114 31507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.60:42103 every 8 connection(s)
I20260809 20:23:05.633452 31430 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-2/data/info.pb
I20260809 20:23:05.637265 31508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:23:05.642117 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 31430
I20260809 20:23:05.642340 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786306623011809-5364-0/minicluster-data/master-2/wal/instance
I20260809 20:23:05.642895 5364 external_mini_cluster.cc:1428] Setting key a80a52d0ca42f7ca9746d7026c004fd5
I20260809 20:23:05.644620 31508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44379 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 20:23:05 dist-test-slave-6k22 krb5kdc[31239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306985, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.60@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
Aug 09 20:23:05 dist-test-slave-6k22 krb5kdc[31239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.60@KRBTEST.COM
I20260809 20:23:05.663656 31513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.647410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51518 (local address 127.5.61.62:44379)
0809 20:23:05.648006 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:05.648011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:05.648667 (+ 656us) server_negotiation.cc:407] Connection header received
0809 20:23:05.648754 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:05.648757 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:05.648816 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:05.648965 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:05.650292 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.651024 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.652650 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.653049 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.653186 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:05.656777 (+ 3591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:05.656782 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:05.656786 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:05.656817 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:05.660296 (+ 3479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.661063 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.661071 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.661075 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.661215 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.661733 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.661738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.661741 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.662108 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:05.662213 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:05.662979 (+ 766us) server_negotiation.cc:299] Negotiation successful
0809 20:23:05.663215 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":322,"threads_started":1}
I20260809 20:23:05.664556 31512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.644792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44754 (local address 127.5.61.60:42103)
0809 20:23:05.645562 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:05.645572 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:05.645608 (+ 36us) server_negotiation.cc:407] Connection header received
0809 20:23:05.647422 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:05.647442 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:05.647966 (+ 524us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:05.648375 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:05.649936 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.651106 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.652451 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.652990 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.653179 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:05.657352 (+ 4173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:05.657365 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:05.657379 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:05.657418 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:05.660961 (+ 3543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.661745 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.661753 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.661756 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.661913 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.662308 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.662311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.662313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.662633 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:05.662942 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:05.663247 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 20:23:05.663481 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":573,"spinlock_wait_cycles":13568,"thread_start_us":359,"threads_started":1}
I20260809 20:23:05.664598 31514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.647410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:44379 (local address 127.0.0.1:51518)
0809 20:23:05.648392 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.648434 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 20:23:05.648605 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:23:05.649032 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:23:05.649038 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:23:05.649664 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:23:05.650109 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:05.650132 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.651250 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.651254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:05.652508 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:05.652520 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.653294 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.653303 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:05.653487 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:05.654172 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:23:05.654193 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:23:05.656576 (+ 2383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:23:05.660441 (+ 3865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:05.660450 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:05.660458 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:05.660922 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:05.661344 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:05.661349 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:05.661353 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:05.661598 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:05.662248 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:23:05.662259 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:23:05.662577 (+ 318us) client_negotiation.cc:769] Sending connection context
0809 20:23:05.662832 (+ 255us) client_negotiation.cc:240] Negotiation successful
0809 20:23:05.663315 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"mutex_wait_us":3,"thread_start_us":599,"threads_started":1}
I20260809 20:23:05.666069 31508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 20:23:05 dist-test-slave-6k22 krb5kdc[31239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306985, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.60@KRBTEST.COM for kudu/127.5.61.61@KRBTEST.COM
Aug 09 20:23:05 dist-test-slave-6k22 krb5kdc[31239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:23:05.679844 31336 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } attempt: 4
I20260809 20:23:05.681231 31514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.666883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:36693 (local address 127.0.0.1:58804)
0809 20:23:05.667146 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.667163 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 20:23:05.667239 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:23:05.667783 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:23:05.667789 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:23:05.668236 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:23:05.668360 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:05.668367 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.669750 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.669757 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:05.670953 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:05.670964 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.671865 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.671869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:05.672074 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:05.672710 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:23:05.672741 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:23:05.674909 (+ 2168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:23:05.678532 (+ 3623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:05.678536 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:05.678538 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:05.678885 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:05.679343 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:05.679347 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:05.679349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:05.679497 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:05.680190 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:23:05.680198 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:23:05.680376 (+ 178us) client_negotiation.cc:769] Sending connection context
0809 20:23:05.680545 (+ 169us) client_negotiation.cc:240] Negotiation successful
0809 20:23:05.680837 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158}
I20260809 20:23:05.682231 31426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.666996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58804 (local address 127.5.61.61:36693)
0809 20:23:05.667179 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:05.667186 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:05.667228 (+ 42us) server_negotiation.cc:407] Connection header received
0809 20:23:05.667371 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:05.667378 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:05.667460 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:05.667585 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:05.668512 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.669598 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.671095 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.671739 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.671864 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:05.675094 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:05.675099 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:05.675102 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:05.675131 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:05.678360 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.679028 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.679032 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.679034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.679226 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.679660 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.679666 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.679670 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.680024 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:05.680194 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:05.681556 (+ 1362us) server_negotiation.cc:299] Negotiation successful
0809 20:23:05.681815 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 20:23:05.683319 31508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } has no permanent_uuid. Determining permanent_uuid...
I20260809 20:23:05.685230 31513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.667518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51522 (local address 127.5.61.62:44379)
0809 20:23:05.667710 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:05.667716 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:05.667734 (+ 18us) server_negotiation.cc:407] Connection header received
0809 20:23:05.667813 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:05.667817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:05.667911 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:05.668030 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:05.669466 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.670467 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.672008 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.672706 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.672835 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:05.677472 (+ 4637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:05.677478 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:05.677483 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:05.677521 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:05.681178 (+ 3657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.682100 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.682104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.682107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.682251 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.682705 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.682710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.682714 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.683030 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:05.683354 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:05.684803 (+ 1449us) server_negotiation.cc:299] Negotiation successful
0809 20:23:05.685007 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
Aug 09 20:23:05 dist-test-slave-6k22 krb5kdc[31239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306984, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for kudu/127.5.61.60@KRBTEST.COM
Aug 09 20:23:05 dist-test-slave-6k22 krb5kdc[31239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306985, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.60@KRBTEST.COM for kudu/127.5.61.60@KRBTEST.COM
I20260809 20:23:05.695307 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.680446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42103 (local address 127.0.0.1:44764)
0809 20:23:05.680621 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.680641 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 20:23:05.680750 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:23:05.681141 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:23:05.681145 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:23:05.681574 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:23:05.681717 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:05.681725 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.683151 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.683158 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:05.684673 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:05.684690 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.685729 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.685735 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:05.685936 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:05.686678 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:23:05.686699 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:23:05.689154 (+ 2455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:23:05.692803 (+ 3649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:05.692809 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:05.692816 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:05.693211 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:05.693601 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:05.693606 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:05.693609 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:05.693754 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:05.694272 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:23:05.694276 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:23:05.694418 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 20:23:05.694591 (+ 173us) client_negotiation.cc:240] Negotiation successful
0809 20:23:05.694899 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 20:23:05.695426 31512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.680678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44764 (local address 127.5.61.60:42103)
0809 20:23:05.680851 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:05.680856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:05.680873 (+ 17us) server_negotiation.cc:407] Connection header received
0809 20:23:05.680917 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:05.680922 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:05.680996 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:05.681090 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:05.681878 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.683031 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.684824 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.685561 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.685703 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:05.689368 (+ 3665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:05.689376 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:05.689385 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:05.689420 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:05.692668 (+ 3248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.693340 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.693346 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.693350 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.693486 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.693869 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.693873 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.693877 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.694145 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:05.694308 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:05.694733 (+ 425us) server_negotiation.cc:299] Negotiation successful
0809 20:23:05.695013 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 20:23:05 dist-test-slave-6k22 krb5kdc[31239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.61@KRBTEST.COM
I20260809 20:23:05.700094 31514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.684018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42103 (local address 127.0.0.1:44780)
0809 20:23:05.684188 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.684211 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 20:23:05.684322 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:23:05.685363 (+ 1041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:23:05.685367 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:23:05.685893 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:23:05.686089 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:05.686101 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.687501 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.687507 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:05.688868 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:05.688884 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.689884 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.689890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:05.690097 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:05.690602 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:23:05.690625 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:23:05.693219 (+ 2594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:23:05.697637 (+ 4418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:05.697643 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:05.697646 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:05.698175 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:05.698597 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:05.698605 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:05.698609 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:05.698751 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:05.699271 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:23:05.699276 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:23:05.699433 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 20:23:05.699627 (+ 194us) client_negotiation.cc:240] Negotiation successful
0809 20:23:05.699867 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 20:23:05.702025 31516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.684251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44780 (local address 127.5.61.60:42103)
0809 20:23:05.684862 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:05.684867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:05.684899 (+ 32us) server_negotiation.cc:407] Connection header received
0809 20:23:05.684935 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:05.684939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:05.684993 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:05.685084 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:05.686266 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.687359 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.689027 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.689730 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.689859 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:05.693712 (+ 3853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:05.693719 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:05.693726 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:05.693764 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:05.697498 (+ 3734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.698287 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.698294 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.698299 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.698475 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.698868 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.698873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.698876 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.699154 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:05.699304 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:05.701433 (+ 2129us) server_negotiation.cc:299] Negotiation successful
0809 20:23:05.701627 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":469,"thread_start_us":281,"threads_started":1}
I20260809 20:23:05.702495 31336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042: Bootstrap starting.
I20260809 20:23:05.704169 31426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.687285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58812 (local address 127.5.61.61:36693)
0809 20:23:05.687438 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:05.687443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:05.687524 (+ 81us) server_negotiation.cc:407] Connection header received
0809 20:23:05.687749 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:05.687756 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:05.687835 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:05.687947 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:05.689382 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.690372 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.692244 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.692699 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.692865 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:05.697685 (+ 4820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:05.697691 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:05.697695 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:05.697731 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:05.701162 (+ 3431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.701950 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.701956 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.701959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.702085 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.702688 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.702693 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.702696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.702990 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:05.703265 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:05.703595 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 20:23:05.703835 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 20:23:05.706286 31336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042: Neither blocks nor log segments found. Creating new log.
I20260809 20:23:05.707154 31508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18670dfab7c04cdca51d63a35a2a1dff: Bootstrap starting.
I20260809 20:23:05.707418 31336 log.cc:824] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042: Log is configured to *not* fsync() on all Append() calls
I20260809 20:23:05.710708 31336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042: No bootstrap required, opened a new log
I20260809 20:23:05.710749 31508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18670dfab7c04cdca51d63a35a2a1dff: Neither blocks nor log segments found. Creating new log.
I20260809 20:23:05.711969 31508 log.cc:824] T 00000000000000000000000000000000 P 18670dfab7c04cdca51d63a35a2a1dff: Log is configured to *not* fsync() on all Append() calls
I20260809 20:23:05.712350 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:23:05.715227 31508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18670dfab7c04cdca51d63a35a2a1dff: No bootstrap required, opened a new log
I20260809 20:23:05.715806 31305 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f89bd7bd61264fc58ed9b63fbf50c042"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51522
I20260809 20:23:05.718906 31336 raft_consensus.cc:348] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44379 } } peers { permanent_uuid: "73053f79361949acb3008794dfa5013c" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } } peers { permanent_uuid: "18670dfab7c04cdca51d63a35a2a1dff" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } }
I20260809 20:23:05.719187 31336 raft_consensus.cc:729] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f89bd7bd61264fc58ed9b63fbf50c042, State: Initialized, Role: FOLLOWER
I20260809 20:23:05.719683 31336 consensus_queue.cc:260] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44379 } } peers { permanent_uuid: "73053f79361949acb3008794dfa5013c" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } } peers { permanent_uuid: "18670dfab7c04cdca51d63a35a2a1dff" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } }
I20260809 20:23:05.720943 31519 sys_catalog.cc:455] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44379 } } peers { permanent_uuid: "73053f79361949acb3008794dfa5013c" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } } peers { permanent_uuid: "18670dfab7c04cdca51d63a35a2a1dff" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } } }
I20260809 20:23:05.721593 31336 sys_catalog.cc:564] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:23:05.721697 31305 raft_consensus.cc:482] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 20:23:05.721899 31305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:23:05.722086 31519 sys_catalog.cc:458] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:23:05.725737 31305 raft_consensus.cc:504] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44379 } } peers { permanent_uuid: "73053f79361949acb3008794dfa5013c" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } } peers { permanent_uuid: "18670dfab7c04cdca51d63a35a2a1dff" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } }
I20260809 20:23:05.726464 31508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18670dfab7c04cdca51d63a35a2a1dff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44379 } } peers { permanent_uuid: "73053f79361949acb3008794dfa5013c" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } } peers { permanent_uuid: "18670dfab7c04cdca51d63a35a2a1dff" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } }
I20260809 20:23:05.726898 31508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18670dfab7c04cdca51d63a35a2a1dff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18670dfab7c04cdca51d63a35a2a1dff, State: Initialized, Role: FOLLOWER
I20260809 20:23:05.728102 31305 leader_election.cc:288] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042 [CANDIDATE]: Term 1 election: Requested vote from peers 73053f79361949acb3008794dfa5013c (127.5.61.61:36693), 18670dfab7c04cdca51d63a35a2a1dff (127.5.61.60:42103)
I20260809 20:23:05.727443 31508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18670dfab7c04cdca51d63a35a2a1dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44379 } } peers { permanent_uuid: "73053f79361949acb3008794dfa5013c" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } } peers { permanent_uuid: "18670dfab7c04cdca51d63a35a2a1dff" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } }
I20260809 20:23:05.735514 31531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18670dfab7c04cdca51d63a35a2a1dff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44379 } } peers { permanent_uuid: "73053f79361949acb3008794dfa5013c" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } } peers { permanent_uuid: "18670dfab7c04cdca51d63a35a2a1dff" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } } }
I20260809 20:23:05.738036 31508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18670dfab7c04cdca51d63a35a2a1dff [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:23:05.739723 31531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18670dfab7c04cdca51d63a35a2a1dff [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:23:05.743722 31477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "18670dfab7c04cdca51d63a35a2a1dff"
I20260809 20:23:05.744199 31477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18670dfab7c04cdca51d63a35a2a1dff [term 0 FOLLOWER]: Advancing to term 1
W20260809 20:23:05.745798 31532 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 20:23:05.745910 31532 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 20:23:05.747107 31477 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 18670dfab7c04cdca51d63a35a2a1dff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f89bd7bd61264fc58ed9b63fbf50c042 in term 1.
I20260809 20:23:05.747821 31275 leader_election.cc:302] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18670dfab7c04cdca51d63a35a2a1dff, f89bd7bd61264fc58ed9b63fbf50c042; no voters:
I20260809 20:23:05.748445 31519 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:23:05.757475 31519 raft_consensus.cc:686] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042 [term 1 LEADER]: Becoming Leader. State: Replica: f89bd7bd61264fc58ed9b63fbf50c042, State: Running, Role: LEADER
I20260809 20:23:05.758179 31519 consensus_queue.cc:237] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44379 } } peers { permanent_uuid: "73053f79361949acb3008794dfa5013c" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } } peers { permanent_uuid: "18670dfab7c04cdca51d63a35a2a1dff" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } }
I20260809 20:23:05.736675 31256 master_runner.cc:186] Error getting master registration for 127.5.61.61:36693: OK, instance_id { permanent_uuid: "73053f79361949acb3008794dfa5013c" instance_seqno: 1786306985083756 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 20:23:05.763314 31256 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 20:23:05.768885 31539 sys_catalog.cc:455] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f89bd7bd61264fc58ed9b63fbf50c042. Latest consensus state: current_term: 1 leader_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44379 } } peers { permanent_uuid: "73053f79361949acb3008794dfa5013c" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } } peers { permanent_uuid: "18670dfab7c04cdca51d63a35a2a1dff" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } } }
I20260809 20:23:05.769349 31539 sys_catalog.cc:458] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042 [sys.catalog]: This master's current role is: LEADER
I20260809 20:23:05.770267 31550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:23:05.773420 31550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260809 20:23:05.791667 31280 tablet.cc:2369] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:23:05.798959 31477 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 18670dfab7c04cdca51d63a35a2a1dff [term 1 FOLLOWER]: Refusing update from remote peer f89bd7bd61264fc58ed9b63fbf50c042: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:23:05.800233 31539 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18670dfab7c04cdca51d63a35a2a1dff" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:23:05.802963 31560 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 20:23:05.804979 31516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.771700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44794 (local address 127.5.61.60:42103)
0809 20:23:05.771886 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:05.771892 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:05.779249 (+ 7357us) server_negotiation.cc:407] Connection header received
0809 20:23:05.779522 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:05.779528 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:05.779669 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:05.779766 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:05.783918 (+ 4152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.785279 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.787760 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.788617 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.788752 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:05.797207 (+ 8455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:05.797213 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:05.797218 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:05.797261 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:05.801283 (+ 4022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.802329 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.802337 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.802344 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.802520 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.803058 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.803064 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.803070 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.803463 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:05.803866 (+ 403us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:05.804391 (+ 525us) server_negotiation.cc:299] Negotiation successful
0809 20:23:05.804621 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 20:23:05.808701 31513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.763745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51526 (local address 127.5.61.62:44379)
0809 20:23:05.763933 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:05.763941 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:05.775534 (+ 11593us) server_negotiation.cc:407] Connection header received
0809 20:23:05.776063 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:05.776069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:05.776148 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:05.776222 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:05.783128 (+ 6906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.791338 (+ 8210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.793257 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.793878 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.793970 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:05.800512 (+ 6542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:05.800518 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:05.800523 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:05.800558 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:05.804496 (+ 3938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.805700 (+ 1204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.805705 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.805708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.805875 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.806556 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.806561 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.806564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.806960 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:05.807246 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:05.808141 (+ 895us) server_negotiation.cc:299] Negotiation successful
0809 20:23:05.808369 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 20:23:05.814637 31426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.765798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58828 (local address 127.5.61.61:36693)
0809 20:23:05.766028 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:05.766036 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:05.779950 (+ 13914us) server_negotiation.cc:407] Connection header received
0809 20:23:05.780044 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:05.780052 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:05.780154 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:05.780277 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:05.785281 (+ 5004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.786817 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.791359 (+ 4542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.792313 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.792435 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:05.796684 (+ 4249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:05.796693 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:05.796697 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:05.796751 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:05.805465 (+ 8714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.811889 (+ 6424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.811894 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.811897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.812056 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.812511 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.812515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.812518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.812887 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:05.813152 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:05.814013 (+ 861us) server_negotiation.cc:299] Negotiation successful
0809 20:23:05.814249 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 20:23:05.818190 31420 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } attempt: 5
I20260809 20:23:05.808784 31430 master_runner.cc:186] Error getting master registration for 127.5.61.61:36693: OK, instance_id { permanent_uuid: "73053f79361949acb3008794dfa5013c" instance_seqno: 1786306985083756 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 20:23:05.820490 31430 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 20:23:05.823923 31531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18670dfab7c04cdca51d63a35a2a1dff [sys.catalog]: SysCatalogTable state changed. Reason: New leader f89bd7bd61264fc58ed9b63fbf50c042. Latest consensus state: current_term: 1 leader_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44379 } } peers { permanent_uuid: "73053f79361949acb3008794dfa5013c" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } } peers { permanent_uuid: "18670dfab7c04cdca51d63a35a2a1dff" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } } }
I20260809 20:23:05.824273 31531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18670dfab7c04cdca51d63a35a2a1dff [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:23:05.824820 31553 mvcc.cc:205] Tried to move back new op lower bound from 7316713413815566336 to 7316713413671735296. Current Snapshot: MvccSnapshot[applied={T|T < 7316713413815566336}]
I20260809 20:23:05.827099 31531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18670dfab7c04cdca51d63a35a2a1dff [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44379 } } peers { permanent_uuid: "73053f79361949acb3008794dfa5013c" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } } peers { permanent_uuid: "18670dfab7c04cdca51d63a35a2a1dff" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } } }
I20260809 20:23:05.827325 31531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18670dfab7c04cdca51d63a35a2a1dff [sys.catalog]: This master's current role is: FOLLOWER
Aug 09 20:23:05 dist-test-slave-6k22 krb5kdc[31239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306985, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.61@KRBTEST.COM for kudu/127.5.61.60@KRBTEST.COM
I20260809 20:23:05.835975 31519 sys_catalog.cc:455] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44379 } } peers { permanent_uuid: "73053f79361949acb3008794dfa5013c" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } } peers { permanent_uuid: "18670dfab7c04cdca51d63a35a2a1dff" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } } }
I20260809 20:23:05.836344 31424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.818682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:42103 (local address 127.0.0.1:44802)
0809 20:23:05.818993 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:05.819012 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 20:23:05.819113 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:23:05.819862 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:23:05.819867 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:23:05.820343 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:23:05.820561 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:05.820576 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.822226 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.822231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:05.823731 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:05.823746 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.825060 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.825066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:05.825274 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:05.825944 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:23:05.825970 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:23:05.828731 (+ 2761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:23:05.833322 (+ 4591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:05.833327 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:05.833331 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:05.833973 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:05.834421 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:05.834426 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:05.834429 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:05.834590 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:05.835357 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:23:05.835362 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:23:05.835517 (+ 155us) client_negotiation.cc:769] Sending connection context
0809 20:23:05.835682 (+ 165us) client_negotiation.cc:240] Negotiation successful
0809 20:23:05.835958 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200}
I20260809 20:23:05.836339 31519 sys_catalog.cc:458] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042 [sys.catalog]: This master's current role is: LEADER
I20260809 20:23:05.837458 31516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:05.819322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44802 (local address 127.5.61.60:42103)
0809 20:23:05.819488 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:05.819498 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:05.819518 (+ 20us) server_negotiation.cc:407] Connection header received
0809 20:23:05.819599 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:05.819604 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:05.819698 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:05.820751 (+ 1053us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:05.820874 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.822093 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.823947 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:05.824602 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:05.828910 (+ 4308us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:05.829156 (+ 246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:05.829164 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:05.829169 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:05.829215 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:05.833170 (+ 3955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.834138 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.834143 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.834146 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.834287 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:05.834813 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:05.834817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:05.834821 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:05.835208 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:05.836743 (+ 1535us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:05.836875 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 20:23:05.837092 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 20:23:05.839290 31550 catalog_manager.cc:1347] Generated new cluster ID: faa09838788b4533a963e31e49335a3a
I20260809 20:23:05.839411 31550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:23:05.843062 31420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73053f79361949acb3008794dfa5013c: Bootstrap starting.
I20260809 20:23:05.847123 31420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73053f79361949acb3008794dfa5013c: Neither blocks nor log segments found. Creating new log.
I20260809 20:23:05.848624 31420 log.cc:824] T 00000000000000000000000000000000 P 73053f79361949acb3008794dfa5013c: Log is configured to *not* fsync() on all Append() calls
I20260809 20:23:05.855305 31420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73053f79361949acb3008794dfa5013c: No bootstrap required, opened a new log
I20260809 20:23:05.867019 31550 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:23:05.867391 31420 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73053f79361949acb3008794dfa5013c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44379 } } peers { permanent_uuid: "73053f79361949acb3008794dfa5013c" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } } peers { permanent_uuid: "18670dfab7c04cdca51d63a35a2a1dff" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } }
I20260809 20:23:05.867918 31420 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73053f79361949acb3008794dfa5013c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73053f79361949acb3008794dfa5013c, State: Initialized, Role: FOLLOWER
I20260809 20:23:05.868466 31420 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73053f79361949acb3008794dfa5013c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44379 } } peers { permanent_uuid: "73053f79361949acb3008794dfa5013c" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } } peers { permanent_uuid: "18670dfab7c04cdca51d63a35a2a1dff" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } }
I20260809 20:23:05.869933 31550 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:23:05.875587 31567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73053f79361949acb3008794dfa5013c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44379 } } peers { permanent_uuid: "73053f79361949acb3008794dfa5013c" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } } peers { permanent_uuid: "18670dfab7c04cdca51d63a35a2a1dff" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } } }
I20260809 20:23:05.877701 31567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73053f79361949acb3008794dfa5013c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:23:05.877565 31420 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73053f79361949acb3008794dfa5013c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:23:05.878041 31389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73053f79361949acb3008794dfa5013c [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:23:05.881963 31389 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 73053f79361949acb3008794dfa5013c [term 1 FOLLOWER]: Refusing update from remote peer f89bd7bd61264fc58ed9b63fbf50c042: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:23:05.883297 31519 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73053f79361949acb3008794dfa5013c" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:23:05.884902 31550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042: Generated new TSK 0
I20260809 20:23:05.885949 31550 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 20:23:05.886667 31390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "73053f79361949acb3008794dfa5013c"
I20260809 20:23:05.914639 31581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73053f79361949acb3008794dfa5013c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44379 } } peers { permanent_uuid: "73053f79361949acb3008794dfa5013c" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } } peers { permanent_uuid: "18670dfab7c04cdca51d63a35a2a1dff" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } } }
I20260809 20:23:05.915066 31581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73053f79361949acb3008794dfa5013c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:23:05.915402 31567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73053f79361949acb3008794dfa5013c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f89bd7bd61264fc58ed9b63fbf50c042. Latest consensus state: current_term: 1 leader_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44379 } } peers { permanent_uuid: "73053f79361949acb3008794dfa5013c" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } } peers { permanent_uuid: "18670dfab7c04cdca51d63a35a2a1dff" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } } }
I20260809 20:23:05.915761 31567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73053f79361949acb3008794dfa5013c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:23:05.916186 31390 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 73053f79361949acb3008794dfa5013c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f89bd7bd61264fc58ed9b63fbf50c042 for term 1 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260809 20:23:05.938910 31584 catalog_manager.cc:1259] Loaded cluster ID: faa09838788b4533a963e31e49335a3a
I20260809 20:23:05.939033 31584 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 73053f79361949acb3008794dfa5013c: loading cluster ID for follower catalog manager: success
I20260809 20:23:05.943224 31584 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 73053f79361949acb3008794dfa5013c: acquiring CA information for follower catalog manager: success
I20260809 20:23:05.945912 31584 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 73053f79361949acb3008794dfa5013c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 20:23:05.993829 31513 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:05.982627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51532 (local address 127.5.61.62:44379)
0809 20:23:05.982847 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:05.982853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:05.988196 (+ 5343us) server_negotiation.cc:407] Connection header received
0809 20:23:05.991980 (+ 3784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:05.991986 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:05.992083 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:05.992159 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:05.993510 (+ 1351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51532: BlockingRecv error: recv got EOF from 127.0.0.1:51532 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260809 20:23:05.995291 31426 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:05.981787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58836 (local address 127.5.61.61:36693)
0809 20:23:05.982012 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:05.982020 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:05.994004 (+ 11984us) server_negotiation.cc:407] Connection header received
0809 20:23:05.994103 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:05.994108 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:05.994188 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:05.994318 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:05.995024 (+ 706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58836: BlockingRecv error: recv got EOF from 127.0.0.1:58836 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260809 20:23:05.996502 31516 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:05.982163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44804 (local address 127.5.61.60:42103)
0809 20:23:05.982325 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:05.982331 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:05.992085 (+ 9754us) server_negotiation.cc:407] Connection header received
0809 20:23:05.994689 (+ 2604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:05.994695 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:05.994782 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:05.994863 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:05.996245 (+ 1382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44804: BlockingRecv error: recv got EOF from 127.0.0.1:44804 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 20:23:06.007826 31426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:06.001018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58850 (local address 127.5.61.61:36693)
0809 20:23:06.001230 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:06.001236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:06.001746 (+ 510us) server_negotiation.cc:407] Connection header received
0809 20:23:06.002031 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:06.002035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:06.002254 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:06.002393 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:23:06.002924 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:06.003761 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:06.005182 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:06.005677 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:06.005829 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:06.006481 (+ 652us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:23:06.007129 (+ 648us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:23:06.007306 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:06.007367 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 20:23:06.007444 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260809 20:23:06.017551 31516 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:06.014600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44818 (local address 127.5.61.60:42103)
0809 20:23:06.014852 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:06.014860 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:06.016147 (+ 1287us) server_negotiation.cc:407] Connection header received
0809 20:23:06.016406 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:06.016410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:06.016504 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:06.016620 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:06.017376 (+ 756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44818: BlockingRecv error: recv got EOF from 127.0.0.1:44818 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 20:23:06.035269 31426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:06.029587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58866 (local address 127.5.61.61:36693)
0809 20:23:06.029860 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:06.029869 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:06.030374 (+ 505us) server_negotiation.cc:407] Connection header received
0809 20:23:06.030580 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:06.030603 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:06.030668 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:06.030767 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:23:06.031181 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:06.031950 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:06.033284 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:06.033805 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:06.033945 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:06.034469 (+ 524us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:23:06.034611 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:23:06.034713 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:06.034858 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 20:23:06.034934 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260809 20:23:06.045382 31516 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:06.042113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44830 (local address 127.5.61.60:42103)
0809 20:23:06.042331 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:06.042339 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:06.044031 (+ 1692us) server_negotiation.cc:407] Connection header received
0809 20:23:06.044211 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:06.044215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:06.044298 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:06.044402 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:06.045142 (+ 740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44830: BlockingRecv error: recv got EOF from 127.0.0.1:44830 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 20:23:06.055506 31426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:06.050091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58874 (local address 127.5.61.61:36693)
0809 20:23:06.050290 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:06.050305 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:06.050818 (+ 513us) server_negotiation.cc:407] Connection header received
0809 20:23:06.051017 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:06.051023 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:06.051091 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:06.051191 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:23:06.051610 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:06.052395 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:06.053741 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:06.054205 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:06.054352 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:06.054738 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:23:06.054842 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:23:06.054943 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:06.055088 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 20:23:06.055172 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 20:23:06.065506 31516 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:06.061622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44844 (local address 127.5.61.60:42103)
0809 20:23:06.061843 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:06.061851 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:06.064097 (+ 2246us) server_negotiation.cc:407] Connection header received
0809 20:23:06.064297 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:06.064303 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:06.064386 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:06.064502 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:06.065250 (+ 748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44844: BlockingRecv error: recv got EOF from 127.0.0.1:44844 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 20:23:06.081324 31426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:06.072759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58882 (local address 127.5.61.61:36693)
0809 20:23:06.072981 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:06.072987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:06.076129 (+ 3142us) server_negotiation.cc:407] Connection header received
0809 20:23:06.076344 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:06.076349 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:06.076445 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:06.076546 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:23:06.077005 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:06.077804 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:06.079361 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:06.080009 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:06.080143 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:06.080539 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:23:06.080645 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:23:06.080743 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:06.080902 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 20:23:06.080981 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
W20260809 20:23:06.089753 31516 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:06.087102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44854 (local address 127.5.61.60:42103)
0809 20:23:06.087445 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:06.087454 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:06.088003 (+ 549us) server_negotiation.cc:407] Connection header received
0809 20:23:06.088205 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:06.088211 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:06.088356 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:06.088451 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:06.089475 (+ 1024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44854: BlockingRecv error: recv got EOF from 127.0.0.1:44854 (error 108)
Metrics: {"server-negotiator.queue_time_us":171}
I20260809 20:23:06.121291 31426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:06.115330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58884 (local address 127.5.61.61:36693)
0809 20:23:06.115605 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:06.115614 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:06.116162 (+ 548us) server_negotiation.cc:407] Connection header received
0809 20:23:06.116491 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:06.116497 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:06.116575 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:06.116705 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:23:06.117167 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:06.117953 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:06.119359 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:06.119916 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:06.120056 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:06.120482 (+ 426us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:23:06.120585 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:23:06.120706 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:06.120855 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 20:23:06.120934 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260809 20:23:06.129483 31516 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:06.127253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44858 (local address 127.5.61.60:42103)
0809 20:23:06.127495 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:06.127504 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:06.128060 (+ 556us) server_negotiation.cc:407] Connection header received
0809 20:23:06.128254 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:06.128259 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:06.128337 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:06.128451 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:06.129226 (+ 775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44858: BlockingRecv error: recv got EOF from 127.0.0.1:44858 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 20:23:06.154487 31426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:06.148931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58900 (local address 127.5.61.61:36693)
0809 20:23:06.149152 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:06.149160 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:06.149705 (+ 545us) server_negotiation.cc:407] Connection header received
0809 20:23:06.149930 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:06.149936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:06.150007 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:06.150108 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:23:06.150494 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:06.151268 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:06.152648 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:06.153128 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:06.153268 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:06.153696 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:23:06.153831 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:23:06.153943 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:06.154095 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 20:23:06.154171 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260809 20:23:06.162447 31516 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:06.160852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44862 (local address 127.5.61.60:42103)
0809 20:23:06.161058 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:06.161065 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:06.161086 (+ 21us) server_negotiation.cc:407] Connection header received
0809 20:23:06.161163 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:06.161168 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:06.161256 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:06.161367 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:06.162216 (+ 849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44862: BlockingRecv error: recv got EOF from 127.0.0.1:44862 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 20:23:06.210721 31426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:06.201688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58904 (local address 127.5.61.61:36693)
0809 20:23:06.201940 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:06.201949 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:06.203096 (+ 1147us) server_negotiation.cc:407] Connection header received
0809 20:23:06.203366 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:06.203376 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:06.203444 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:06.203557 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:23:06.204034 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:06.205238 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:06.208072 (+ 2834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:06.208906 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:06.209020 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:06.209838 (+ 818us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:23:06.209983 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:23:06.210071 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:06.210363 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 20:23:06.210467 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 20:23:06.225281 31516 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:06.222930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44878 (local address 127.5.61.60:42103)
0809 20:23:06.223170 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:06.223179 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:06.223787 (+ 608us) server_negotiation.cc:407] Connection header received
0809 20:23:06.224003 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:06.224009 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:06.224096 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:06.224210 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:06.225028 (+ 818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44878: BlockingRecv error: recv got EOF from 127.0.0.1:44878 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 20:23:06.304901 31426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:06.297320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58920 (local address 127.5.61.61:36693)
0809 20:23:06.297527 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:06.297535 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:06.298155 (+ 620us) server_negotiation.cc:407] Connection header received
0809 20:23:06.298421 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:06.298428 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:06.298535 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:06.298655 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:23:06.299227 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:06.300559 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:06.302679 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:06.303375 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:06.303519 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:06.304096 (+ 577us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:23:06.304267 (+ 171us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:23:06.304365 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:06.304544 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 20:23:06.304628 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260809 20:23:06.316296 31516 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:06.313835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44892 (local address 127.5.61.60:42103)
0809 20:23:06.314047 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:06.314056 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:06.314660 (+ 604us) server_negotiation.cc:407] Connection header received
0809 20:23:06.314884 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:06.314890 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:06.314987 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:06.315094 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:06.316038 (+ 944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44892: BlockingRecv error: recv got EOF from 127.0.0.1:44892 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 20:23:06.454361 31426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:06.448181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58936 (local address 127.5.61.61:36693)
0809 20:23:06.448406 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:06.448412 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:06.448819 (+ 407us) server_negotiation.cc:407] Connection header received
0809 20:23:06.449030 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:06.449033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:06.449099 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:06.449177 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:23:06.449710 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:06.450433 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:06.452576 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:06.452999 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:06.453089 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:06.453593 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:23:06.453685 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:23:06.453750 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:06.453978 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 20:23:06.454052 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
W20260809 20:23:06.462158 31516 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:06.459973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44900 (local address 127.5.61.60:42103)
0809 20:23:06.460173 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:06.460179 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:06.460534 (+ 355us) server_negotiation.cc:407] Connection header received
0809 20:23:06.460822 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:06.460829 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:06.460914 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:06.461035 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:06.461903 (+ 868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44900: BlockingRecv error: recv got EOF from 127.0.0.1:44900 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 20:23:06.468370 31690 sys_catalog.cc:455] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89bd7bd61264fc58ed9b63fbf50c042" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 44379 } } peers { permanent_uuid: "73053f79361949acb3008794dfa5013c" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 36693 } } peers { permanent_uuid: "18670dfab7c04cdca51d63a35a2a1dff" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 42103 } } }
I20260809 20:23:06.468752 31690 sys_catalog.cc:458] T 00000000000000000000000000000000 P f89bd7bd61264fc58ed9b63fbf50c042 [sys.catalog]: This master's current role is: LEADER
I20260809 20:23:06.727216 31426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:06.722250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58946 (local address 127.5.61.61:36693)
0809 20:23:06.722498 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:06.722505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:06.722738 (+ 233us) server_negotiation.cc:407] Connection header received
0809 20:23:06.722923 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:06.722928 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:06.722998 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:06.723081 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:23:06.723442 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:06.724262 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:06.725681 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:06.726071 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:06.726165 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:06.726519 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:23:06.726615 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:23:06.726681 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:06.726837 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 20:23:06.726932 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260809 20:23:06.734774 31516 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:06.732958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44916 (local address 127.5.61.60:42103)
0809 20:23:06.733167 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:06.733173 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:06.733288 (+ 115us) server_negotiation.cc:407] Connection header received
0809 20:23:06.733521 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:06.733531 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:06.733617 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:06.733722 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:06.734559 (+ 837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44916: BlockingRecv error: recv got EOF from 127.0.0.1:44916 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 20:23:06.807533 31560 catalog_manager.cc:1259] Loaded cluster ID: faa09838788b4533a963e31e49335a3a
I20260809 20:23:06.807704 31560 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 18670dfab7c04cdca51d63a35a2a1dff: loading cluster ID for follower catalog manager: success
I20260809 20:23:06.810262 31560 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 18670dfab7c04cdca51d63a35a2a1dff: acquiring CA information for follower catalog manager: success
I20260809 20:23:06.812153 31560 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 18670dfab7c04cdca51d63a35a2a1dff: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 20:23:07.258172 31721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:07.253408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58954 (local address 127.5.61.61:36693)
0809 20:23:07.253931 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:07.253935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:07.253956 (+ 21us) server_negotiation.cc:407] Connection header received
0809 20:23:07.254085 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:07.254089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:07.254161 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:07.254260 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:23:07.254662 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:07.255383 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:07.256770 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:07.257153 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:07.257243 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:07.257583 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:23:07.257676 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:23:07.257755 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:07.257865 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 20:23:07.257925 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":297,"threads_started":1}
I20260809 20:23:07.270249 31729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:07.264239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44922 (local address 127.5.61.60:42103)
0809 20:23:07.264731 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:07.264738 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:07.264824 (+ 86us) server_negotiation.cc:407] Connection header received
0809 20:23:07.265065 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:07.265075 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:07.265230 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:07.265379 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:23:07.265808 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:07.266499 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:07.267948 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:07.268372 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:07.268553 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:07.269075 (+ 522us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:23:07.269634 (+ 559us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:23:07.269817 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:07.269882 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 20:23:07.269971 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":239,"threads_started":1}
I20260809 20:23:07.284889 31729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:07.276269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44934 (local address 127.5.61.60:42103)
0809 20:23:07.277586 (+ 1317us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:07.277599 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:07.278597 (+ 998us) server_negotiation.cc:407] Connection header received
0809 20:23:07.278878 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:07.278886 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:07.278984 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:07.279047 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:23:07.279914 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:07.280798 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:07.282723 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:07.283303 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:07.283454 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:07.283947 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:23:07.284146 (+ 199us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:23:07.284237 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:07.284472 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 20:23:07.284554 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1124}
I20260809 20:23:07.285516 31721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:07.276753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58964 (local address 127.5.61.61:36693)
0809 20:23:07.278655 (+ 1902us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:07.278664 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:07.278691 (+ 27us) server_negotiation.cc:407] Connection header received
0809 20:23:07.279021 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:07.279030 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:07.279113 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:07.279177 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:23:07.280210 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:07.281334 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:07.283421 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:07.283977 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:07.284130 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:07.284679 (+ 549us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:23:07.284815 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:23:07.284883 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:07.285113 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 20:23:07.285195 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1720}
I20260809 20:23:07.285941 31735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:07.276270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51548 (local address 127.5.61.62:44379)
0809 20:23:07.279076 (+ 2806us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:07.279080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:07.279095 (+ 15us) server_negotiation.cc:407] Connection header received
0809 20:23:07.279183 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:07.279187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:07.279349 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:07.279609 (+ 260us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:23:07.280379 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:07.281456 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:07.283127 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:07.283798 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:07.283975 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:07.284719 (+ 744us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:23:07.285206 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:23:07.285371 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:07.285558 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 20:23:07.285655 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2703,"thread_start_us":2595,"threads_started":1}
I20260809 20:23:07.291049 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 31256
I20260809 20:23:07.292224 31330 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:23:07.404234 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 31256
W20260809 20:23:07.429090 31365 connection.cc:535] server connection from 127.0.0.1:58790 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 20:23:07.429658 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 31345
I20260809 20:23:07.430725 31414 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:23:07.539239 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 31345
I20260809 20:23:07.563243 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 31430
I20260809 20:23:07.564531 31501 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:23:07.671999 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 31430
2026-08-09T20:23:07Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5083 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 20:23:07 dist-test-slave-6k22 krb5kdc[31744](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 20:23:07 dist-test-slave-6k22 krb5kdc[31744](info): set up 2 sockets
Aug 09 20:23:07 dist-test-slave-6k22 krb5kdc[31744](info): commencing operation
krb5kdc: starting...
W20260809 20:23:08.950894 5364 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.003s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/krb5kdc/test-user.keytab.
Aug 09 20:23:09 dist-test-slave-6k22 krb5kdc[31744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T20:23:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T20:23:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.61.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
Entry for principal kudu/127.5.61.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab.
I20260809 20:23:09.260118 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:37621
--webserver_interface=127.5.61.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:45671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:37621,127.5.61.61:44465,127.5.61.60:43393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 20:23:09.603444 31759 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 20:23:09.603871 31759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:23:09.603997 31759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:23:09.604136 31759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:23:09.611203 31759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:23:09.611285 31759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:23:09.611344 31759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:23:09.611421 31759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:23:09.611515 31759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:23:09.621063 31759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:45671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:37621,127.5.61.61:44465,127.5.61.60:43393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.62.keytab
--principal=kudu/127.5.61.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.62:37621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.61.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:23:09.622273 31759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:23:09.623639 31759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:23:09.636052 31766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:23:09.635996 31768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:23:09.636083 31770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:23:09.636879 31759 server_base.cc:1034] running on GCE node
I20260809 20:23:09.637603 31759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:23:09.638935 31759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:23:09.640179 31759 hybrid_clock.cc:648] HybridClock initialized: now 1786306989640131 us; error 49 us; skew 500 ppm
I20260809 20:23:09.640460 31759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:23:09 dist-test-slave-6k22 krb5kdc[31744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306989, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:23:09.643529 31759 init.cc:377] Logged in from keytab as kudu/127.5.61.62@KRBTEST.COM (short username kudu)
I20260809 20:23:09.644881 31759 webserver.cc:460] Webserver started at http://127.5.61.62:46437/ using document root <none> and password file <none>
I20260809 20:23:09.645498 31759 fs_manager.cc:359] Metadata directory not provided
I20260809 20:23:09.645581 31759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:23:09.645854 31759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:23:09.648778 31759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-0/data/instance:
uuid: "69164fe54c7a4ea598e2611f4c1b43fb"
format_stamp: "Formatted at 2026-08-09 20:23:09 on dist-test-slave-6k22"
server_key: "5ebd3557325fb44bc0fb4dd7a81db27a"
server_key_iv: "55d73bc9cbe30cd007b50a3a2695a366"
server_key_version: "encryptionkey@0"
I20260809 20:23:09.649586 31759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-0/wal/instance:
uuid: "69164fe54c7a4ea598e2611f4c1b43fb"
format_stamp: "Formatted at 2026-08-09 20:23:09 on dist-test-slave-6k22"
server_key: "5ebd3557325fb44bc0fb4dd7a81db27a"
server_key_iv: "55d73bc9cbe30cd007b50a3a2695a366"
server_key_version: "encryptionkey@0"
I20260809 20:23:09.654424 31759 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 20:23:09.657790 31777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:23:09.659077 31759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 20:23:09.659255 31759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-0/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-0/wal
uuid: "69164fe54c7a4ea598e2611f4c1b43fb"
format_stamp: "Formatted at 2026-08-09 20:23:09 on dist-test-slave-6k22"
server_key: "5ebd3557325fb44bc0fb4dd7a81db27a"
server_key_iv: "55d73bc9cbe30cd007b50a3a2695a366"
server_key_version: "encryptionkey@0"
I20260809 20:23:09.659466 31759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:23:09.686636 31759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:23:09.690927 31759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:23:09.691151 31759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:23:09.709486 31759 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.62:37621
I20260809 20:23:09.709517 31836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.62:37621 every 8 connection(s)
I20260809 20:23:09.710806 31759 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-0/data/info.pb
I20260809 20:23:09.714363 31837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:23:09.720536 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 31759
I20260809 20:23:09.720726 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-0/wal/instance
I20260809 20:23:09.721195 5364 external_mini_cluster.cc:1428] Setting key 74971f7d18759e61ead167fd82379850
I20260809 20:23:09.720731 31837 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37621 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 20:23:09 dist-test-slave-6k22 krb5kdc[31744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306989, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:23:09.743489 31840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:09.725110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46986 (local address 127.5.61.62:37621)
0809 20:23:09.725891 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:09.725898 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:09.725944 (+ 46us) server_negotiation.cc:407] Connection header received
0809 20:23:09.726987 (+ 1043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:09.726993 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:09.727209 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:09.727416 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:09.728666 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:09.729797 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:09.731157 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:09.731771 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:09.731966 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:09.735595 (+ 3629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:09.735609 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:09.735625 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:09.735670 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:09.739691 (+ 4021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:09.740361 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:09.740371 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:09.740377 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:09.740578 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:09.740943 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:09.740950 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:09.740955 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:09.741593 (+ 638us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:09.741852 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:09.742327 (+ 475us) server_negotiation.cc:299] Negotiation successful
0809 20:23:09.742720 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":83,"server-negotiator.queue_time_us":573,"thread_start_us":412,"threads_started":1}
I20260809 20:23:09.743513 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:09.723883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:37621 (local address 127.0.0.1:46986)
0809 20:23:09.724621 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:09.724665 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 20:23:09.726632 (+ 1967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:23:09.727439 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:23:09.727445 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:23:09.728126 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:23:09.728457 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:09.728473 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:09.729972 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:09.729977 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:09.730994 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:09.731006 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:09.731999 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:09.732007 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:09.732196 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:09.732865 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:23:09.732884 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:23:09.735312 (+ 2428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:23:09.739829 (+ 4517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:09.739839 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:09.739847 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:09.740221 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:09.740690 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:09.740692 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:09.740695 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:09.740835 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:09.741722 (+ 887us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:23:09.741727 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:23:09.741934 (+ 207us) client_negotiation.cc:769] Sending connection context
0809 20:23:09.742129 (+ 195us) client_negotiation.cc:240] Negotiation successful
0809 20:23:09.742516 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":325,"threads_started":1}
I20260809 20:23:09.746893 31837 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } has no permanent_uuid. Determining permanent_uuid...
I20260809 20:23:09.747908 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:09.747533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:44465 (local address 127.0.0.1:37114)
0809 20:23:09.747713 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:09.747811 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:44465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260809 20:23:09.748145 31779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.61.61:44465: connect: Connection refused (error 111)
I20260809 20:23:09.749651 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:09.749317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:44465 (local address 127.0.0.1:37116)
0809 20:23:09.749468 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:09.749530 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:44465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 20:23:09.749822 31837 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.61:44465: Network error: Client connection negotiation failed: client connection to 127.5.61.61:44465: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.5.61.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.61.keytab.
Entry for principal kudu/127.5.61.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.61.keytab.
I20260809 20:23:09.793097 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.61:44465
--webserver_interface=127.5.61.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.61.keytab
--principal=kudu/127.5.61.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:45671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:37621,127.5.61.61:44465,127.5.61.60:43393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 20:23:09.799109 31837 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } attempt: 1
I20260809 20:23:09.800328 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:09.799861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:44465 (local address 127.0.0.1:37122)
0809 20:23:09.800056 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:09.800188 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:44465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 20:23:09.801370 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:09.801023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:44465 (local address 127.0.0.1:37138)
0809 20:23:09.801147 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:09.801214 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:44465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 20:23:09.801558 31837 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.61:44465: Network error: Client connection negotiation failed: client connection to 127.5.61.61:44465: connect: Connection refused (error 111)
I20260809 20:23:09.869813 31837 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } attempt: 2
I20260809 20:23:09.871073 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:09.870619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:44465 (local address 127.0.0.1:37154)
0809 20:23:09.870822 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:09.870925 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:44465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 20:23:09.872195 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:09.871814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:44465 (local address 127.0.0.1:37162)
0809 20:23:09.871977 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:09.872069 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:44465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 20:23:09.872391 31837 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.61:44465: Network error: Client connection negotiation failed: client connection to 127.5.61.61:44465: connect: Connection refused (error 111)
I20260809 20:23:09.963693 31837 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } attempt: 3
I20260809 20:23:09.964984 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:09.964535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:44465 (local address 127.0.0.1:37178)
0809 20:23:09.964741 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:09.964845 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:44465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 20:23:09.966058 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:09.965696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:44465 (local address 127.0.0.1:37182)
0809 20:23:09.965844 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:09.965925 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:44465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 20:23:09.966253 31837 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.61:44465: Network error: Client connection negotiation failed: client connection to 127.5.61.61:44465: connect: Connection refused (error 111)
I20260809 20:23:10.109553 31837 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } attempt: 4
I20260809 20:23:10.111039 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.110441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:44465 (local address 127.0.0.1:37194)
0809 20:23:10.110790 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.110913 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:44465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179}
I20260809 20:23:10.112265 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.111813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:44465 (local address 127.0.0.1:37202)
0809 20:23:10.112070 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.112152 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.61:44465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
W20260809 20:23:10.112532 31837 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.61:44465: Network error: Client connection negotiation failed: client connection to 127.5.61.61:44465: connect: Connection refused (error 111)
W20260809 20:23:10.132642 31844 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 20:23:10.132970 31844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:23:10.133068 31844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:23:10.133222 31844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:23:10.140404 31844 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:23:10.140484 31844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:23:10.140615 31844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:23:10.140702 31844 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:23:10.140827 31844 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:23:10.150578 31844 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:45671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:37621,127.5.61.61:44465,127.5.61.60:43393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.61.keytab
--principal=kudu/127.5.61.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.61:44465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.61.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:23:10.151823 31844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:23:10.153112 31844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:23:10.165292 31852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:23:10.165326 31853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:23:10.165364 31855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:23:10.166114 31844 server_base.cc:1034] running on GCE node
I20260809 20:23:10.166932 31844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:23:10.168318 31844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:23:10.169570 31844 hybrid_clock.cc:648] HybridClock initialized: now 1786306990169478 us; error 52 us; skew 500 ppm
I20260809 20:23:10.169914 31844 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:23:10 dist-test-slave-6k22 krb5kdc[31744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306990, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:23:10.173300 31844 init.cc:377] Logged in from keytab as kudu/127.5.61.61@KRBTEST.COM (short username kudu)
I20260809 20:23:10.177394 31844 webserver.cc:460] Webserver started at http://127.5.61.61:33287/ using document root <none> and password file <none>
I20260809 20:23:10.178290 31844 fs_manager.cc:359] Metadata directory not provided
I20260809 20:23:10.178444 31844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:23:10.178825 31844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:23:10.182050 31844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-1/data/instance:
uuid: "475a45de10604d4086a89e1bedf508e0"
format_stamp: "Formatted at 2026-08-09 20:23:10 on dist-test-slave-6k22"
server_key: "444c5e23d5d69e66dc280d0f67ea9730"
server_key_iv: "966facdc278c84822fdf7b7c83e68129"
server_key_version: "encryptionkey@0"
I20260809 20:23:10.182778 31844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-1/wal/instance:
uuid: "475a45de10604d4086a89e1bedf508e0"
format_stamp: "Formatted at 2026-08-09 20:23:10 on dist-test-slave-6k22"
server_key: "444c5e23d5d69e66dc280d0f67ea9730"
server_key_iv: "966facdc278c84822fdf7b7c83e68129"
server_key_version: "encryptionkey@0"
I20260809 20:23:10.187924 31844 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260809 20:23:10.191196 31864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:23:10.192512 31844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 20:23:10.192685 31844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-1/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-1/wal
uuid: "475a45de10604d4086a89e1bedf508e0"
format_stamp: "Formatted at 2026-08-09 20:23:10 on dist-test-slave-6k22"
server_key: "444c5e23d5d69e66dc280d0f67ea9730"
server_key_iv: "966facdc278c84822fdf7b7c83e68129"
server_key_version: "encryptionkey@0"
I20260809 20:23:10.192900 31844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:23:10.242233 31844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:23:10.246773 31844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:23:10.247069 31844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:23:10.265569 31844 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.61:44465
I20260809 20:23:10.265592 31922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.61:44465 every 8 connection(s)
I20260809 20:23:10.266927 31844 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-1/data/info.pb
I20260809 20:23:10.270637 31923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:23:10.272526 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 31844
I20260809 20:23:10.272723 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-1/wal/instance
I20260809 20:23:10.273190 5364 external_mini_cluster.cc:1428] Setting key 6e667409fffcb44cf60227254dc0bd1a
I20260809 20:23:10.277164 31923 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37621 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 20:23:10 dist-test-slave-6k22 krb5kdc[31744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306990, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.61@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:23:10.301172 31925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.279729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46988 (local address 127.5.61.62:37621)
0809 20:23:10.280253 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:10.280259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:10.281305 (+ 1046us) server_negotiation.cc:407] Connection header received
0809 20:23:10.284180 (+ 2875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:10.284184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:10.284265 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:10.284364 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:10.286461 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.287197 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.288807 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.289588 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.289745 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:10.293562 (+ 3817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:10.293571 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:10.293578 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:10.293639 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:10.297623 (+ 3984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:10.298385 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:10.298393 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:10.298400 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:10.298569 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:10.298976 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:10.298984 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:10.298990 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:10.299370 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:10.299540 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:10.300334 (+ 794us) server_negotiation.cc:299] Negotiation successful
0809 20:23:10.300620 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":258,"threads_started":1}
I20260809 20:23:10.301580 31926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.280281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:37621 (local address 127.0.0.1:46988)
0809 20:23:10.281111 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.281172 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 20:23:10.283872 (+ 2700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:23:10.284627 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:23:10.284640 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:23:10.285456 (+ 816us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:23:10.286244 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:10.286272 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.287384 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.287388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:10.288641 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:10.288653 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.289800 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.289804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:10.290003 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:10.290797 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:23:10.290819 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:23:10.293324 (+ 2505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:23:10.297761 (+ 4437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:10.297767 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:10.297779 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:10.298218 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:10.298679 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:10.298682 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:10.298684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:10.298844 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:10.299490 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:23:10.299495 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:23:10.299881 (+ 386us) client_negotiation.cc:769] Sending connection context
0809 20:23:10.300128 (+ 247us) client_negotiation.cc:240] Negotiation successful
0809 20:23:10.300490 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":326,"threads_started":1}
I20260809 20:23:10.303411 31923 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.61.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.61.60@KRBTEST.COM" created.
Aug 09 20:23:10 dist-test-slave-6k22 krb5kdc[31744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306990, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.61@KRBTEST.COM for kudu/127.5.61.61@KRBTEST.COM
I20260809 20:23:10.319312 31926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.304289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:44465 (local address 127.0.0.1:37210)
0809 20:23:10.304456 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.304477 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 20:23:10.304605 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:23:10.305910 (+ 1305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:23:10.305919 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:23:10.306446 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:23:10.306673 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:10.306691 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.308198 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.308207 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:10.309386 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:10.309403 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.310265 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.310270 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:10.310386 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:10.310979 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:23:10.311027 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:23:10.313599 (+ 2572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:23:10.316818 (+ 3219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:10.316826 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:10.316830 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:10.317182 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:10.317570 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:10.317577 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:10.317580 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:10.317717 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:10.318332 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:23:10.318337 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:23:10.318501 (+ 164us) client_negotiation.cc:769] Sending connection context
0809 20:23:10.318642 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 20:23:10.318929 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 20:23:10.319607 31928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.304711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37210 (local address 127.5.61.61:44465)
0809 20:23:10.305428 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:10.305434 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:10.305459 (+ 25us) server_negotiation.cc:407] Connection header received
0809 20:23:10.305500 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:10.305506 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:10.305713 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:10.305832 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:10.306885 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.308013 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.309532 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.310069 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.310172 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:10.313817 (+ 3645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:10.313828 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:10.313841 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:10.313869 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:10.316677 (+ 2808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:10.317326 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:10.317335 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:10.317338 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:10.317461 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:10.317831 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:10.317838 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:10.317842 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:10.318173 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:10.318428 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:10.318785 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 20:23:10.319040 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":572,"thread_start_us":420,"threads_started":1}
I20260809 20:23:10.322019 31923 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } has no permanent_uuid. Determining permanent_uuid...
I20260809 20:23:10.323194 31926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.322770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:43393 (local address 127.0.0.1:40264)
0809 20:23:10.322941 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.323056 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260809 20:23:10.323594 31867 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
I20260809 20:23:10.325121 31926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.324819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:43393 (local address 127.0.0.1:40276)
0809 20:23:10.324956 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.325030 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260809 20:23:10.325392 31923 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:43393: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.61.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.60.keytab.
Entry for principal kudu/127.5.61.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.60.keytab.
I20260809 20:23:10.352280 5364 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
/tmp/dist-test-task7uf_Cz/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.60:43393
--webserver_interface=127.5.61.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.60.keytab
--principal=kudu/127.5.61.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.61.20:45671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.61.62:37621,127.5.61.61:44465,127.5.61.60:43393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 20:23:10.374720 31923 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } attempt: 1
I20260809 20:23:10.376014 31926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.375492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:43393 (local address 127.0.0.1:40292)
0809 20:23:10.375726 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.375844 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260809 20:23:10.377136 31926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.376759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:43393 (local address 127.0.0.1:40304)
0809 20:23:10.376911 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.377008 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 20:23:10.377379 31923 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:43393: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
I20260809 20:23:10.411811 31837 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } attempt: 5
Aug 09 20:23:10 dist-test-slave-6k22 krb5kdc[31744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306989, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for kudu/127.5.61.61@KRBTEST.COM
I20260809 20:23:10.426829 31928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.412833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37218 (local address 127.5.61.61:44465)
0809 20:23:10.413061 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:10.413066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:10.413087 (+ 21us) server_negotiation.cc:407] Connection header received
0809 20:23:10.413271 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:10.413277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:10.413344 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:10.413461 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:10.414287 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.415016 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.416325 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.416767 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.416901 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:10.421116 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:10.421121 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:10.421124 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:10.421158 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:10.423955 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:10.424899 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:10.424903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:10.424905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:10.425028 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:10.425487 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:10.425491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:10.425492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:10.425770 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:10.425883 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:10.426426 (+ 543us) server_negotiation.cc:299] Negotiation successful
0809 20:23:10.426592 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 20:23:10.427038 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.412693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:44465 (local address 127.0.0.1:37218)
0809 20:23:10.412936 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.412959 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 20:23:10.413102 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:23:10.413522 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:23:10.413527 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:23:10.413940 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:23:10.414121 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:10.414131 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.415170 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.415175 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:10.416156 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:10.416170 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.416930 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.416936 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:10.417122 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:10.417873 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:23:10.417890 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:23:10.420880 (+ 2990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:23:10.424108 (+ 3228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:10.424114 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:10.424144 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:10.424769 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:10.425159 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:10.425165 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:10.425169 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:10.425363 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:10.425902 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:23:10.425908 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:23:10.426090 (+ 182us) client_negotiation.cc:769] Sending connection context
0809 20:23:10.426329 (+ 239us) client_negotiation.cc:240] Negotiation successful
0809 20:23:10.426639 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260809 20:23:10.428277 31837 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } has no permanent_uuid. Determining permanent_uuid...
I20260809 20:23:10.429562 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.429086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:43393 (local address 127.0.0.1:40314)
0809 20:23:10.429348 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.429425 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260809 20:23:10.430740 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.430316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:43393 (local address 127.0.0.1:40320)
0809 20:23:10.430498 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.430595 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260809 20:23:10.431003 31837 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:43393: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
I20260809 20:23:10.445613 31923 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } attempt: 2
I20260809 20:23:10.446723 31926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.446284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:43393 (local address 127.0.0.1:40334)
0809 20:23:10.446470 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.446562 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 20:23:10.447880 31926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.447518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:43393 (local address 127.0.0.1:40346)
0809 20:23:10.447694 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.447767 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260809 20:23:10.448150 31923 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:43393: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
I20260809 20:23:10.482257 31837 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } attempt: 1
I20260809 20:23:10.483743 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.483151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:43393 (local address 127.0.0.1:40352)
0809 20:23:10.483396 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.483488 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260809 20:23:10.484851 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.484521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:43393 (local address 127.0.0.1:40362)
0809 20:23:10.484659 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.484727 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 20:23:10.485107 31837 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:43393: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
I20260809 20:23:10.539414 31923 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } attempt: 3
I20260809 20:23:10.540741 31926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.540269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:43393 (local address 127.0.0.1:40374)
0809 20:23:10.540490 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.540581 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 20:23:10.541931 31926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.541514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:43393 (local address 127.0.0.1:40378)
0809 20:23:10.541744 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.541822 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260809 20:23:10.542136 31923 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:43393: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
I20260809 20:23:10.553308 31837 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } attempt: 2
I20260809 20:23:10.554481 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.554027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:43393 (local address 127.0.0.1:40384)
0809 20:23:10.554231 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.554320 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 20:23:10.555521 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.555210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:43393 (local address 127.0.0.1:40390)
0809 20:23:10.555358 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.555425 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 20:23:10.555781 31837 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:43393: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
I20260809 20:23:10.662026 31837 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } attempt: 3
I20260809 20:23:10.663532 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.662906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:43393 (local address 127.0.0.1:40392)
0809 20:23:10.663250 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.663365 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260809 20:23:10.664780 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.664356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:43393 (local address 127.0.0.1:40396)
0809 20:23:10.664563 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.664644 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260809 20:23:10.664997 31837 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:43393: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
I20260809 20:23:10.685343 31923 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } attempt: 4
I20260809 20:23:10.686446 31926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.685964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:43393 (local address 127.0.0.1:40404)
0809 20:23:10.686220 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.686301 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 20:23:10.687501 31926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.687188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:43393 (local address 127.0.0.1:40408)
0809 20:23:10.687340 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.687403 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 20:23:10.687824 31923 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.61.60:43393: Network error: Client connection negotiation failed: client connection to 127.5.61.60:43393: connect: Connection refused (error 111)
W20260809 20:23:10.725317 31931 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 20:23:10.725710 31931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 20:23:10.725848 31931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 20:23:10.725992 31931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 20:23:10.733596 31931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 20:23:10.733755 31931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 20:23:10.733901 31931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 20:23:10.734001 31931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 20:23:10.734131 31931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 20:23:10.744521 31931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.61.20:45671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.61.62:37621,127.5.61.61:44465,127.5.61.60:43393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/krb5kdc/kudu_127.5.61.60.keytab
--principal=kudu/127.5.61.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.61.60:43393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.61.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 18:55:23 UTC on ccycloud.quasar-joeceo.root.comops.site
build id 4576685
ASAN enabled
I20260809 20:23:10.745873 31931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 20:23:10.747191 31931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 20:23:10.759809 31940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:23:10.760075 31937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 20:23:10.760231 31936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 20:23:10.761114 31931 server_base.cc:1034] running on GCE node
I20260809 20:23:10.761997 31931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 20:23:10.763535 31931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 20:23:10.764824 31931 hybrid_clock.cc:648] HybridClock initialized: now 1786306990764747 us; error 49 us; skew 500 ppm
I20260809 20:23:10.765136 31931 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 20:23:10 dist-test-slave-6k22 krb5kdc[31744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306990, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 20:23:10.773331 31931 init.cc:377] Logged in from keytab as kudu/127.5.61.60@KRBTEST.COM (short username kudu)
I20260809 20:23:10.774799 31931 webserver.cc:460] Webserver started at http://127.5.61.60:44109/ using document root <none> and password file <none>
I20260809 20:23:10.775465 31931 fs_manager.cc:359] Metadata directory not provided
I20260809 20:23:10.775602 31931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 20:23:10.775935 31931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 20:23:10.778924 31931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-2/data/instance:
uuid: "670371545df54050a9fa9e969612855d"
format_stamp: "Formatted at 2026-08-09 20:23:10 on dist-test-slave-6k22"
server_key: "913f4b02315b48db38e3c4a630a5c517"
server_key_iv: "4aa689b469e616a7c37f2031fba8b982"
server_key_version: "encryptionkey@0"
I20260809 20:23:10.779699 31931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-2/wal/instance:
uuid: "670371545df54050a9fa9e969612855d"
format_stamp: "Formatted at 2026-08-09 20:23:10 on dist-test-slave-6k22"
server_key: "913f4b02315b48db38e3c4a630a5c517"
server_key_iv: "4aa689b469e616a7c37f2031fba8b982"
server_key_version: "encryptionkey@0"
I20260809 20:23:10.784672 31931 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 20:23:10.787966 31949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 20:23:10.789594 31931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 20:23:10.789760 31931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-2/data,/tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-2/wal
uuid: "670371545df54050a9fa9e969612855d"
format_stamp: "Formatted at 2026-08-09 20:23:10 on dist-test-slave-6k22"
server_key: "913f4b02315b48db38e3c4a630a5c517"
server_key_iv: "4aa689b469e616a7c37f2031fba8b982"
server_key_version: "encryptionkey@0"
I20260809 20:23:10.790004 31931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 20:23:10.814109 31931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 20:23:10.818468 31931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 20:23:10.818749 31931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 20:23:10.837754 31931 rpc_server.cc:307] RPC server started. Bound to: 127.5.61.60:43393
I20260809 20:23:10.837741 32010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.61.60:43393 every 8 connection(s)
I20260809 20:23:10.839298 31931 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-2/data/info.pb
I20260809 20:23:10.842214 31837 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } attempt: 4
I20260809 20:23:10.843437 32011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 20:23:10.846424 5364 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu as pid 31931
I20260809 20:23:10.846630 5364 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7uf_Cz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786306623011809-5364-0/minicluster-data/master-2/wal/instance
I20260809 20:23:10.847124 5364 external_mini_cluster.cc:1428] Setting key bb1561281b7162f112c9ee8c1a8fef3d
I20260809 20:23:10.854355 32011 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37621 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 20:23:10 dist-test-slave-6k22 krb5kdc[31744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306989, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.62@KRBTEST.COM for kudu/127.5.61.60@KRBTEST.COM
I20260809 20:23:10.866535 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.842887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:43393 (local address 127.0.0.1:40424)
0809 20:23:10.843081 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.843106 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 20:23:10.843267 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:23:10.847953 (+ 4686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:23:10.847959 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:23:10.848495 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:23:10.848729 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:10.848742 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.851831 (+ 3089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.851839 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:10.853264 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:10.853283 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.854730 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.854736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:10.854929 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:10.855640 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:23:10.855662 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:23:10.858595 (+ 2933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:23:10.862854 (+ 4259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:10.862861 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:10.862864 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:10.863442 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:10.864118 (+ 676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:10.864124 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:10.864128 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:10.864334 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:10.865381 (+ 1047us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:23:10.865387 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:23:10.865575 (+ 188us) client_negotiation.cc:769] Sending connection context
0809 20:23:10.865724 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 20:23:10.866034 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 20:23:10.868067 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.843933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40424 (local address 127.5.61.60:43393)
0809 20:23:10.844707 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:10.844721 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:10.844771 (+ 50us) server_negotiation.cc:407] Connection header received
0809 20:23:10.846913 (+ 2142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:10.846942 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:10.847460 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:10.848984 (+ 1524us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:10.849478 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.851630 (+ 2152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.853474 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.854520 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.855323 (+ 803us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:10.859014 (+ 3691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:10.859029 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:10.859053 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:10.859112 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:10.862705 (+ 3593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:10.863721 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:10.863730 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:10.863738 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:10.863963 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:10.864451 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:10.864456 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:10.864459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:10.865211 (+ 752us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:10.865409 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:10.866269 (+ 860us) server_negotiation.cc:299] Negotiation successful
0809 20:23:10.866890 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":330,"threads_started":1}
Aug 09 20:23:10 dist-test-slave-6k22 krb5kdc[31744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.60@KRBTEST.COM
Aug 09 20:23:10 dist-test-slave-6k22 krb5kdc[31744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306990, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.60@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:23:10.875753 31837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb: Bootstrap starting.
I20260809 20:23:10.877950 32015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.848062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40428 (local address 127.5.61.60:43393)
0809 20:23:10.858248 (+ 10186us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:10.858254 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:10.858279 (+ 25us) server_negotiation.cc:407] Connection header received
0809 20:23:10.858331 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:10.858338 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:10.858452 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:10.858567 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:10.861141 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.862974 (+ 1833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.864787 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.865264 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.865476 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:10.871474 (+ 5998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:10.871478 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:10.871481 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:10.871507 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:10.874756 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:10.875631 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:10.875636 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:10.875639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:10.875804 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:10.876293 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:10.876298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:10.876306 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:10.876671 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:10.876974 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:10.877310 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 20:23:10.877529 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":10038,"thread_start_us":9882,"threads_started":1}
I20260809 20:23:10.880124 31837 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb: Neither blocks nor log segments found. Creating new log.
I20260809 20:23:10.881431 32016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.858569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.62:37621 (local address 127.0.0.1:47002)
0809 20:23:10.863466 (+ 4897us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.863511 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 20:23:10.863761 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:23:10.864345 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:23:10.864351 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:23:10.865211 (+ 860us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:23:10.865688 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:10.865701 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.867215 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.867224 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:10.868996 (+ 1772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:10.869009 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.869983 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.869989 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:10.870197 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:10.871773 (+ 1576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:23:10.871795 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:23:10.874320 (+ 2525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:23:10.878262 (+ 3942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:10.878274 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:10.878281 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:10.878857 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:10.879273 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:10.879278 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:10.879281 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:10.879444 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:10.880316 (+ 872us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:23:10.880324 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:23:10.880534 (+ 210us) client_negotiation.cc:769] Sending connection context
0809 20:23:10.880735 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 20:23:10.881019 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4638,"mutex_wait_us":29,"thread_start_us":4505,"threads_started":1}
I20260809 20:23:10.881556 31837 log.cc:824] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb: Log is configured to *not* fsync() on all Append() calls
I20260809 20:23:10.882493 32018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.858764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47002 (local address 127.5.61.62:37621)
0809 20:23:10.862902 (+ 4138us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:10.862908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:10.863626 (+ 718us) server_negotiation.cc:407] Connection header received
0809 20:23:10.863950 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:10.863955 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:10.864032 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:10.864103 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:10.865841 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.867008 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.869150 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.869734 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.869908 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:10.874516 (+ 4608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:10.874522 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:10.874531 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:10.874558 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:10.878117 (+ 3559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:10.878993 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:10.878999 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:10.879002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:10.879123 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:10.879636 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:10.879641 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:10.879644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:10.880121 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:10.881663 (+ 1542us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:10.881814 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 20:23:10.882044 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2270,"thread_start_us":1465,"threads_started":1}
I20260809 20:23:10.883718 32011 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } has no permanent_uuid. Determining permanent_uuid...
I20260809 20:23:10.885032 31837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb: No bootstrap required, opened a new log
Aug 09 20:23:10 dist-test-slave-6k22 krb5kdc[31744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306990, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.60@KRBTEST.COM for kudu/127.5.61.61@KRBTEST.COM
Aug 09 20:23:10 dist-test-slave-6k22 krb5kdc[31744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.62@KRBTEST.COM
I20260809 20:23:10.896003 31837 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37621 } } peers { permanent_uuid: "475a45de10604d4086a89e1bedf508e0" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } } peers { permanent_uuid: "670371545df54050a9fa9e969612855d" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } }
I20260809 20:23:10.896368 31837 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69164fe54c7a4ea598e2611f4c1b43fb, State: Initialized, Role: FOLLOWER
I20260809 20:23:10.896991 31837 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37621 } } peers { permanent_uuid: "475a45de10604d4086a89e1bedf508e0" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } } peers { permanent_uuid: "670371545df54050a9fa9e969612855d" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } }
I20260809 20:23:10.898639 32016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.884376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.61:44465 (local address 127.0.0.1:37222)
0809 20:23:10.884600 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.884616 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 20:23:10.884706 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:23:10.885066 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:23:10.885070 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:23:10.885414 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:23:10.885537 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:10.885545 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.886505 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.886511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:10.887374 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:10.887385 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.888116 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.888122 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:10.888321 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:10.888882 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:23:10.888908 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:23:10.891645 (+ 2737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:23:10.895532 (+ 3887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:10.895538 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:10.895542 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:10.896396 (+ 854us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:10.896865 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:10.896871 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:10.896875 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:10.897026 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:10.897620 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:23:10.897625 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:23:10.897849 (+ 224us) client_negotiation.cc:769] Sending connection context
0809 20:23:10.897994 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 20:23:10.898264 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113}
I20260809 20:23:10.898694 31928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.884486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37222 (local address 127.5.61.61:44465)
0809 20:23:10.884668 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:10.884674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:10.884693 (+ 19us) server_negotiation.cc:407] Connection header received
0809 20:23:10.884834 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:10.884840 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:10.884910 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:10.885027 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:10.885689 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.886346 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.887524 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.887963 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.888106 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:10.891848 (+ 3742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:10.891855 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:10.891867 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:10.891911 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:10.895390 (+ 3479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:10.896517 (+ 1127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:10.896527 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:10.896549 (+ 22us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:10.896751 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:10.897140 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:10.897145 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:10.897149 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:10.897501 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:10.897659 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:10.898089 (+ 430us) server_negotiation.cc:299] Negotiation successful
0809 20:23:10.898310 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 20:23:10.900174 32011 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } has no permanent_uuid. Determining permanent_uuid...
I20260809 20:23:10.900364 32022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37621 } } peers { permanent_uuid: "475a45de10604d4086a89e1bedf508e0" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } } peers { permanent_uuid: "670371545df54050a9fa9e969612855d" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } } }
I20260809 20:23:10.900840 32022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:23:10.901733 31837 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:23:10.903007 32019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.880049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47010 (local address 127.5.61.62:37621)
0809 20:23:10.880674 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:10.880678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:10.883350 (+ 2672us) server_negotiation.cc:407] Connection header received
0809 20:23:10.883403 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:10.883407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:10.883475 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:10.883593 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:10.885477 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.886748 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.888631 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.889166 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.889315 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:10.894199 (+ 4884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:10.894204 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:10.894207 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:10.894242 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:10.898168 (+ 3926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:10.899281 (+ 1113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:10.899290 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:10.899296 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:10.899667 (+ 371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:10.900401 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:10.900407 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:10.900410 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:10.900939 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:10.901225 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:10.902327 (+ 1102us) server_negotiation.cc:299] Negotiation successful
0809 20:23:10.902555 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":342,"threads_started":1}
Aug 09 20:23:10 dist-test-slave-6k22 krb5kdc[31744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306990, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.60@KRBTEST.COM for kudu/127.5.61.60@KRBTEST.COM
Aug 09 20:23:10 dist-test-slave-6k22 krb5kdc[31744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.61.61@KRBTEST.COM
I20260809 20:23:10.918569 32015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.901335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40438 (local address 127.5.61.60:43393)
0809 20:23:10.901539 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:10.901545 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:10.901564 (+ 19us) server_negotiation.cc:407] Connection header received
0809 20:23:10.901709 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:10.901714 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:10.901803 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:10.901898 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:10.902969 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.904037 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.905684 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.906589 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.906750 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:10.910738 (+ 3988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:10.910747 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:10.910754 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:10.910812 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:10.915085 (+ 4273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:10.916009 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:10.916014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:10.916017 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:10.916168 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:10.916638 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:10.916645 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:10.916648 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:10.917104 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:10.917294 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:10.917924 (+ 630us) server_negotiation.cc:299] Negotiation successful
0809 20:23:10.918169 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 20:23:10.919806 32016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.900973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:43393 (local address 127.0.0.1:40438)
0809 20:23:10.901445 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.901465 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 20:23:10.901579 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:23:10.902061 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:23:10.902067 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:23:10.902586 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:23:10.902806 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:10.902820 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.904236 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.904242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:10.905491 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:10.905508 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.906797 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.906807 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:10.907086 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:10.907821 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:23:10.907856 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:23:10.910525 (+ 2669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:23:10.915276 (+ 4751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:10.915283 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:10.915295 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:10.915878 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:10.916306 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:10.916311 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:10.916314 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:10.916486 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:10.917359 (+ 873us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:23:10.917365 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:23:10.917613 (+ 248us) client_negotiation.cc:769] Sending connection context
0809 20:23:10.919053 (+ 1440us) client_negotiation.cc:240] Negotiation successful
0809 20:23:10.919370 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332}
W20260809 20:23:10.921453 32033 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 20:23:10.921640 32033 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 20:23:10.926585 32011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 670371545df54050a9fa9e969612855d: Bootstrap starting.
I20260809 20:23:10.930969 32011 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 670371545df54050a9fa9e969612855d: Neither blocks nor log segments found. Creating new log.
I20260809 20:23:10.931898 31928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.905449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37236 (local address 127.5.61.61:44465)
0809 20:23:10.905635 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:10.905642 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:10.905863 (+ 221us) server_negotiation.cc:407] Connection header received
0809 20:23:10.906059 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:10.906065 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:10.906143 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:10.906270 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:10.908437 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.909579 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.911275 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.912022 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.912208 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:10.924295 (+ 12087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:10.924303 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:10.924309 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:10.924349 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:10.927957 (+ 3608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:10.928822 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:10.928827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:10.928831 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:10.929007 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:10.929443 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:10.929448 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:10.929451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:10.929782 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:10.930064 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:10.931269 (+ 1205us) server_negotiation.cc:299] Negotiation successful
0809 20:23:10.931511 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 20:23:10.932484 32011 log.cc:824] T 00000000000000000000000000000000 P 670371545df54050a9fa9e969612855d: Log is configured to *not* fsync() on all Append() calls
I20260809 20:23:10.921190 31759 master_runner.cc:186] Error getting master registration for 127.5.61.61:44465: OK, instance_id { permanent_uuid: "475a45de10604d4086a89e1bedf508e0" instance_seqno: 1786306990260246 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 20:23:10.936409 31759 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 20:23:10.937315 5364 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 20:23:10.938678 32011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 670371545df54050a9fa9e969612855d: No bootstrap required, opened a new log
I20260809 20:23:10.940476 31807 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "69164fe54c7a4ea598e2611f4c1b43fb"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47010
I20260809 20:23:10.940944 31807 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 20:23:10.941145 31807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb [term 0 FOLLOWER]: Advancing to term 1
W20260809 20:23:10.944007 31784 tablet.cc:2369] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:23:10.945907 31807 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37621 } } peers { permanent_uuid: "475a45de10604d4086a89e1bedf508e0" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } } peers { permanent_uuid: "670371545df54050a9fa9e969612855d" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } }
I20260809 20:23:10.948819 31807 leader_election.cc:288] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb [CANDIDATE]: Term 1 election: Requested vote from peers 475a45de10604d4086a89e1bedf508e0 (127.5.61.61:44465), 670371545df54050a9fa9e969612855d (127.5.61.60:43393)
I20260809 20:23:10.950583 32011 raft_consensus.cc:348] T 00000000000000000000000000000000 P 670371545df54050a9fa9e969612855d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37621 } } peers { permanent_uuid: "475a45de10604d4086a89e1bedf508e0" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } } peers { permanent_uuid: "670371545df54050a9fa9e969612855d" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } }
I20260809 20:23:10.951195 32011 raft_consensus.cc:729] T 00000000000000000000000000000000 P 670371545df54050a9fa9e969612855d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 670371545df54050a9fa9e969612855d, State: Initialized, Role: FOLLOWER
I20260809 20:23:10.951840 32011 consensus_queue.cc:260] T 00000000000000000000000000000000 P 670371545df54050a9fa9e969612855d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37621 } } peers { permanent_uuid: "475a45de10604d4086a89e1bedf508e0" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } } peers { permanent_uuid: "670371545df54050a9fa9e969612855d" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } }
I20260809 20:23:10.954694 32039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 670371545df54050a9fa9e969612855d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37621 } } peers { permanent_uuid: "475a45de10604d4086a89e1bedf508e0" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } } peers { permanent_uuid: "670371545df54050a9fa9e969612855d" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } } }
I20260809 20:23:10.955873 32039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 670371545df54050a9fa9e969612855d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:23:10.955834 31980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "670371545df54050a9fa9e969612855d"
I20260809 20:23:10.956348 31980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 670371545df54050a9fa9e969612855d [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:23:10.955828 32011 sys_catalog.cc:564] T 00000000000000000000000000000000 P 670371545df54050a9fa9e969612855d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:23:10.959823 31980 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 670371545df54050a9fa9e969612855d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69164fe54c7a4ea598e2611f4c1b43fb in term 1.
I20260809 20:23:10.960959 31780 leader_election.cc:302] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 670371545df54050a9fa9e969612855d, 69164fe54c7a4ea598e2611f4c1b43fb; no voters:
I20260809 20:23:10.961462 32022 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 20:23:10.976061 32022 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb [term 1 LEADER]: Becoming Leader. State: Replica: 69164fe54c7a4ea598e2611f4c1b43fb, State: Running, Role: LEADER
I20260809 20:23:10.976737 32022 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37621 } } peers { permanent_uuid: "475a45de10604d4086a89e1bedf508e0" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } } peers { permanent_uuid: "670371545df54050a9fa9e969612855d" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } }
I20260809 20:23:10.980857 32040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69164fe54c7a4ea598e2611f4c1b43fb. Latest consensus state: current_term: 1 leader_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37621 } } peers { permanent_uuid: "475a45de10604d4086a89e1bedf508e0" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } } peers { permanent_uuid: "670371545df54050a9fa9e969612855d" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } } }
I20260809 20:23:10.981356 32040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb [sys.catalog]: This master's current role is: LEADER
I20260809 20:23:10.987035 31923 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } attempt: 5
Aug 09 20:23:11 dist-test-slave-6k22 krb5kdc[31744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786306990, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.61.61@KRBTEST.COM for kudu/127.5.61.60@KRBTEST.COM
I20260809 20:23:11.017060 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.987963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40466 (local address 127.5.61.60:43393)
0809 20:23:10.988157 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:10.988164 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:10.988900 (+ 736us) server_negotiation.cc:407] Connection header received
0809 20:23:10.988989 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:10.988998 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:10.989103 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:10.989212 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:10.990673 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.992215 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.995359 (+ 3144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.996301 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.996450 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:11.007299 (+ 10849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:11.007308 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:11.007316 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:11.007379 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:11.012225 (+ 4846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:11.013333 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:11.013342 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:11.013347 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:11.013538 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:11.014074 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:11.014082 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:11.014087 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:11.014568 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:11.014768 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:11.015777 (+ 1009us) server_negotiation.cc:299] Negotiation successful
0809 20:23:11.016427 (+ 650us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 20:23:11.018281 31926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.988230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.61.60:43393 (local address 127.0.0.1:40466)
0809 20:23:10.988566 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0809 20:23:10.988592 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 20:23:10.988761 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 20:23:10.989463 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 20:23:10.989471 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 20:23:10.990177 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0809 20:23:10.990444 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:10.990461 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.992417 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.992426 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:10.995068 (+ 2642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 20:23:10.995088 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 20:23:10.996828 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 20:23:10.996837 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 20:23:10.997114 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:11.003115 (+ 6001us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 20:23:11.003148 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 20:23:11.007032 (+ 3884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 20:23:11.012449 (+ 5417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:11.012457 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:11.012464 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:11.013158 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:11.013707 (+ 549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 20:23:11.013715 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 20:23:11.013721 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 20:23:11.013920 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 20:23:11.014853 (+ 933us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 20:23:11.014861 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 20:23:11.015059 (+ 198us) client_negotiation.cc:769] Sending connection context
0809 20:23:11.015219 (+ 160us) client_negotiation.cc:240] Negotiation successful
0809 20:23:11.015516 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187}
I20260809 20:23:11.020627 32015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.981978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40452 (local address 127.5.61.60:43393)
0809 20:23:10.982183 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:10.982189 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:10.992417 (+ 10228us) server_negotiation.cc:407] Connection header received
0809 20:23:10.993513 (+ 1096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:10.993519 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:10.993595 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:10.993677 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:11.004690 (+ 11013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:11.005750 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:11.007604 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:11.008373 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:11.008503 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:11.012574 (+ 4071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:11.012581 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:11.012587 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:11.012626 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:11.016219 (+ 3593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:11.017412 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:11.017418 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:11.017421 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:11.017590 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:11.018426 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:11.018431 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:11.018434 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:11.018814 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:11.019064 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:11.020061 (+ 997us) server_negotiation.cc:299] Negotiation successful
0809 20:23:11.020273 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 20:23:11.026522 31923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 475a45de10604d4086a89e1bedf508e0: Bootstrap starting.
I20260809 20:23:11.035507 31923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 475a45de10604d4086a89e1bedf508e0: Neither blocks nor log segments found. Creating new log.
I20260809 20:23:11.038251 31923 log.cc:824] T 00000000000000000000000000000000 P 475a45de10604d4086a89e1bedf508e0: Log is configured to *not* fsync() on all Append() calls
I20260809 20:23:11.040051 32047 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 20:23:11.045647 32047 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 20:23:11.055414 31928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.991406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37250 (local address 127.5.61.61:44465)
0809 20:23:10.991581 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:10.991586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:11.023655 (+ 32069us) server_negotiation.cc:407] Connection header received
0809 20:23:11.024279 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:11.024285 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:11.024375 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:11.024450 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:11.027737 (+ 3287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:11.028976 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:11.033037 (+ 4061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:11.033783 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:11.033883 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:11.046702 (+ 12819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:11.046710 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:11.046715 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:11.046757 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:11.050714 (+ 3957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:11.052467 (+ 1753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:11.052473 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:11.052477 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:11.052630 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:11.053304 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:11.053309 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:11.053313 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:11.053699 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:11.053934 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:11.054753 (+ 819us) server_negotiation.cc:299] Negotiation successful
0809 20:23:11.055023 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 20:23:11.063861 31923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 475a45de10604d4086a89e1bedf508e0: No bootstrap required, opened a new log
I20260809 20:23:11.064323 32019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:10.982011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47020 (local address 127.5.61.62:37621)
0809 20:23:10.982300 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:10.982308 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:10.994193 (+ 11885us) server_negotiation.cc:407] Connection header received
0809 20:23:10.994280 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:10.994288 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:10.994406 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:10.994523 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:11.001473 (+ 6950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:11.002818 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:11.040429 (+ 37611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:11.041442 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:11.041574 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:11.049912 (+ 8338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 20:23:11.049921 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 20:23:11.049928 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 20:23:11.049980 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 20:23:11.054745 (+ 4765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:11.059122 (+ 4377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:11.059131 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:11.059137 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:11.059316 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 20:23:11.061089 (+ 1773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 20:23:11.061097 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 20:23:11.061101 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 20:23:11.061567 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 20:23:11.061874 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:11.063484 (+ 1610us) server_negotiation.cc:299] Negotiation successful
0809 20:23:11.063812 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
W20260809 20:23:11.070823 31955 tablet.cc:2369] T 00000000000000000000000000000000 P 670371545df54050a9fa9e969612855d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 20:23:11.078683 31980 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 670371545df54050a9fa9e969612855d [term 1 FOLLOWER]: Refusing update from remote peer 69164fe54c7a4ea598e2611f4c1b43fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:23:11.080183 32022 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "670371545df54050a9fa9e969612855d" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:23:11.080309 31923 raft_consensus.cc:348] T 00000000000000000000000000000000 P 475a45de10604d4086a89e1bedf508e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37621 } } peers { permanent_uuid: "475a45de10604d4086a89e1bedf508e0" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } } peers { permanent_uuid: "670371545df54050a9fa9e969612855d" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } }
I20260809 20:23:11.080681 31923 raft_consensus.cc:729] T 00000000000000000000000000000000 P 475a45de10604d4086a89e1bedf508e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 475a45de10604d4086a89e1bedf508e0, State: Initialized, Role: FOLLOWER
I20260809 20:23:11.081236 31923 consensus_queue.cc:260] T 00000000000000000000000000000000 P 475a45de10604d4086a89e1bedf508e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37621 } } peers { permanent_uuid: "475a45de10604d4086a89e1bedf508e0" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } } peers { permanent_uuid: "670371545df54050a9fa9e969612855d" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } }
I20260809 20:23:11.092190 32056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 475a45de10604d4086a89e1bedf508e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37621 } } peers { permanent_uuid: "475a45de10604d4086a89e1bedf508e0" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } } peers { permanent_uuid: "670371545df54050a9fa9e969612855d" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } } }
I20260809 20:23:11.092724 32056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 475a45de10604d4086a89e1bedf508e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:23:11.093614 31923 sys_catalog.cc:564] T 00000000000000000000000000000000 P 475a45de10604d4086a89e1bedf508e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 20:23:11.094089 31892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 475a45de10604d4086a89e1bedf508e0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 20:23:11.098412 31892 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 475a45de10604d4086a89e1bedf508e0 [term 1 FOLLOWER]: Refusing update from remote peer 69164fe54c7a4ea598e2611f4c1b43fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 20:23:11.099010 31893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "475a45de10604d4086a89e1bedf508e0"
I20260809 20:23:11.101006 32022 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "475a45de10604d4086a89e1bedf508e0" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 20:23:11.102531 31893 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 475a45de10604d4086a89e1bedf508e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69164fe54c7a4ea598e2611f4c1b43fb in term 1.
I20260809 20:23:11.114005 32056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 475a45de10604d4086a89e1bedf508e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69164fe54c7a4ea598e2611f4c1b43fb. Latest consensus state: current_term: 1 leader_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37621 } } peers { permanent_uuid: "475a45de10604d4086a89e1bedf508e0" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } } peers { permanent_uuid: "670371545df54050a9fa9e969612855d" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } } }
I20260809 20:23:11.114418 32056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 475a45de10604d4086a89e1bedf508e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:23:11.116516 32054 mvcc.cc:205] Tried to move back new op lower bound from 7316713435442892800 to 7316713435046379520. Current Snapshot: MvccSnapshot[applied={T|T < 7316713435442892800}]
I20260809 20:23:11.120591 32064 mvcc.cc:205] Tried to move back new op lower bound from 7316713435442892800 to 7316713435046379520. Current Snapshot: MvccSnapshot[applied={T|T < 7316713435442892800}]
I20260809 20:23:11.121248 32056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 475a45de10604d4086a89e1bedf508e0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37621 } } peers { permanent_uuid: "475a45de10604d4086a89e1bedf508e0" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } } peers { permanent_uuid: "670371545df54050a9fa9e969612855d" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } } }
I20260809 20:23:11.121557 32056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 475a45de10604d4086a89e1bedf508e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:23:11.132350 32040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37621 } } peers { permanent_uuid: "475a45de10604d4086a89e1bedf508e0" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } } peers { permanent_uuid: "670371545df54050a9fa9e969612855d" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } } }
I20260809 20:23:11.132759 32040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb [sys.catalog]: This master's current role is: LEADER
I20260809 20:23:11.134470 32047 catalog_manager.cc:1347] Generated new cluster ID: 058a3b604dba4078959eccb1b2d1edd2
I20260809 20:23:11.134557 32047 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 20:23:11.140203 32039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 670371545df54050a9fa9e969612855d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69164fe54c7a4ea598e2611f4c1b43fb. Latest consensus state: current_term: 1 leader_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37621 } } peers { permanent_uuid: "475a45de10604d4086a89e1bedf508e0" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } } peers { permanent_uuid: "670371545df54050a9fa9e969612855d" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } } }
I20260809 20:23:11.140595 32039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 670371545df54050a9fa9e969612855d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 20:23:11.146638 32022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37621 } } peers { permanent_uuid: "475a45de10604d4086a89e1bedf508e0" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } } peers { permanent_uuid: "670371545df54050a9fa9e969612855d" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } } }
I20260809 20:23:11.147912 32022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb [sys.catalog]: This master's current role is: LEADER
I20260809 20:23:11.151769 32059 mvcc.cc:205] Tried to move back new op lower bound from 7316713435442892800 to 7316713435046379520. Current Snapshot: MvccSnapshot[applied={T|T < 7316713435442892800}]
I20260809 20:23:11.172814 32039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 670371545df54050a9fa9e969612855d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69164fe54c7a4ea598e2611f4c1b43fb" member_type: VOTER last_known_addr { host: "127.5.61.62" port: 37621 } } peers { permanent_uuid: "475a45de10604d4086a89e1bedf508e0" member_type: VOTER last_known_addr { host: "127.5.61.61" port: 44465 } } peers { permanent_uuid: "670371545df54050a9fa9e969612855d" member_type: VOTER last_known_addr { host: "127.5.61.60" port: 43393 } } }
I20260809 20:23:11.173084 32039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 670371545df54050a9fa9e969612855d [sys.catalog]: This master's current role is: FOLLOWER
W20260809 20:23:11.191282 32080 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 670371545df54050a9fa9e969612855d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 20:23:11.191479 32080 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 20:23:11.184767 31931 master_runner.cc:186] Error getting master registration for 127.5.61.61:44465: OK, instance_id { permanent_uuid: "475a45de10604d4086a89e1bedf508e0" instance_seqno: 1786306990260246 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 20:23:11.200076 31931 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 20:23:11.228008 32089 catalog_manager.cc:1259] Loaded cluster ID: 058a3b604dba4078959eccb1b2d1edd2
I20260809 20:23:11.228235 32089 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 475a45de10604d4086a89e1bedf508e0: loading cluster ID for follower catalog manager: success
W20260809 20:23:11.229507 32089 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 475a45de10604d4086a89e1bedf508e0: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260809 20:23:11.229724 32089 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260809 20:23:11.232285 32047 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 20:23:11.234108 32047 catalog_manager.cc:1504] Loading token signing keys...
I20260809 20:23:11.252585 32047 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69164fe54c7a4ea598e2611f4c1b43fb: Generated new TSK 0
I20260809 20:23:11.253686 32047 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 20:23:11.390177 5364 mini_kdc.cc:348] Time spent kdestroy: real 0.113s user 0.000s sys 0.029s
W20260809 20:23:11.436251 32019 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:11.427434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47026 (local address 127.5.61.62:37621)
0809 20:23:11.427708 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:11.427716 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:11.433768 (+ 6052us) server_negotiation.cc:407] Connection header received
0809 20:23:11.434133 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:11.434140 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:11.434361 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:11.434450 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:11.435921 (+ 1471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47026: BlockingRecv error: recv got EOF from 127.0.0.1:47026 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 20:23:11.449717 31928 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:11.434344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37262 (local address 127.5.61.61:44465)
0809 20:23:11.434561 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:11.434567 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:11.447618 (+ 13051us) server_negotiation.cc:407] Connection header received
0809 20:23:11.447983 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:11.447993 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:11.448099 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:11.448185 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:11.449439 (+ 1254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37262: BlockingRecv error: recv got EOF from 127.0.0.1:37262 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260809 20:23:11.451551 32015 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:11.435442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40474 (local address 127.5.61.60:43393)
0809 20:23:11.435637 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:11.435643 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:11.449365 (+ 13722us) server_negotiation.cc:407] Connection header received
0809 20:23:11.449834 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:11.449839 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:11.449927 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:11.450023 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:11.451300 (+ 1277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40474: BlockingRecv error: recv got EOF from 127.0.0.1:40474 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 20:23:11.490127 31928 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:11.478979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37270 (local address 127.5.61.61:44465)
0809 20:23:11.479183 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:11.479189 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:11.487831 (+ 8642us) server_negotiation.cc:407] Connection header received
0809 20:23:11.488414 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:11.488420 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:11.488587 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:11.488671 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:11.489822 (+ 1151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37270: BlockingRecv error: recv got EOF from 127.0.0.1:37270 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 20:23:11.539232 31928 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:11.526167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37272 (local address 127.5.61.61:44465)
0809 20:23:11.526362 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:11.526368 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:11.536980 (+ 10612us) server_negotiation.cc:407] Connection header received
0809 20:23:11.537555 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:11.537562 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:11.537700 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:11.537798 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:11.538975 (+ 1177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37272: BlockingRecv error: recv got EOF from 127.0.0.1:37272 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260809 20:23:11.594786 31928 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:11.581330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37288 (local address 127.5.61.61:44465)
0809 20:23:11.581539 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:11.581546 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:11.592206 (+ 10660us) server_negotiation.cc:407] Connection header received
0809 20:23:11.592789 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:11.592795 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:11.592923 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:11.593032 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:11.594471 (+ 1439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37288: BlockingRecv error: recv got EOF from 127.0.0.1:37288 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260809 20:23:11.646726 31928 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:11.633823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37294 (local address 127.5.61.61:44465)
0809 20:23:11.634066 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:11.634072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:11.644636 (+ 10564us) server_negotiation.cc:407] Connection header received
0809 20:23:11.645137 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:11.645143 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:11.645255 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:11.645343 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:11.646430 (+ 1087us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37294: BlockingRecv error: recv got EOF from 127.0.0.1:37294 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
W20260809 20:23:11.706398 31928 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:11.691510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37306 (local address 127.5.61.61:44465)
0809 20:23:11.691711 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:11.691717 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:11.704574 (+ 12857us) server_negotiation.cc:407] Connection header received
0809 20:23:11.705041 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:11.705047 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:11.705131 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:11.705208 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:11.706148 (+ 940us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37306: BlockingRecv error: recv got EOF from 127.0.0.1:37306 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260809 20:23:11.743494 31928 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:11.741075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37318 (local address 127.5.61.61:44465)
0809 20:23:11.741294 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:11.741306 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:11.741981 (+ 675us) server_negotiation.cc:407] Connection header received
0809 20:23:11.742233 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:11.742242 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:11.742315 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:11.742430 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:11.743237 (+ 807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37318: BlockingRecv error: recv got EOF from 127.0.0.1:37318 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260809 20:23:11.781255 31928 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:11.778897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37328 (local address 127.5.61.61:44465)
0809 20:23:11.779137 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:11.779150 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:11.779739 (+ 589us) server_negotiation.cc:407] Connection header received
0809 20:23:11.780042 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:11.780055 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:11.780129 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:11.780244 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:11.780998 (+ 754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37328: BlockingRecv error: recv got EOF from 127.0.0.1:37328 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260809 20:23:11.850944 31928 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:11.848722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37330 (local address 127.5.61.61:44465)
0809 20:23:11.849001 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:11.849010 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:11.849645 (+ 635us) server_negotiation.cc:407] Connection header received
0809 20:23:11.849811 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:11.849817 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:11.849905 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:11.850018 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:11.850735 (+ 717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37330: BlockingRecv error: recv got EOF from 127.0.0.1:37330 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260809 20:23:11.984417 31928 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 20:23:11.982560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37332 (local address 127.5.61.61:44465)
0809 20:23:11.982799 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:11.982803 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:11.983122 (+ 319us) server_negotiation.cc:407] Connection header received
0809 20:23:11.983290 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:11.983294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:11.983375 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:11.983469 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 20:23:11.984222 (+ 753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37332: BlockingRecv error: recv got EOF from 127.0.0.1:37332 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
I20260809 20:23:12.193423 32080 catalog_manager.cc:1259] Loaded cluster ID: 058a3b604dba4078959eccb1b2d1edd2
I20260809 20:23:12.193602 32080 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 670371545df54050a9fa9e969612855d: loading cluster ID for follower catalog manager: success
I20260809 20:23:12.196483 32080 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 670371545df54050a9fa9e969612855d: acquiring CA information for follower catalog manager: success
I20260809 20:23:12.198246 32080 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 670371545df54050a9fa9e969612855d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 20:23:12.232955 32089 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 475a45de10604d4086a89e1bedf508e0: acquiring CA information for follower catalog manager: success
I20260809 20:23:12.235037 32089 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 475a45de10604d4086a89e1bedf508e0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 20:23:12.251209 31928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:12.245426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37342 (local address 127.5.61.61:44465)
0809 20:23:12.245654 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:12.245661 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:12.245977 (+ 316us) server_negotiation.cc:407] Connection header received
0809 20:23:12.246207 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:12.246213 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:12.246372 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:12.246516 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:23:12.246962 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:12.247807 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:12.249155 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:12.249523 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:12.249655 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:12.250095 (+ 440us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:23:12.250559 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:23:12.250715 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:12.250815 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 20:23:12.250907 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 20:23:12.262343 32165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:12.256641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40476 (local address 127.5.61.60:43393)
0809 20:23:12.257171 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:12.257179 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:12.257197 (+ 18us) server_negotiation.cc:407] Connection header received
0809 20:23:12.257398 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:12.257402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:12.257535 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:12.257704 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:23:12.258124 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:12.258836 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:12.260272 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:12.260698 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:12.260846 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:12.261330 (+ 484us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:23:12.261769 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:23:12.261934 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:12.261988 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 20:23:12.262064 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":280,"threads_started":1}
I20260809 20:23:12.280441 32165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:12.273149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40482 (local address 127.5.61.60:43393)
0809 20:23:12.273358 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:12.273364 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:12.274430 (+ 1066us) server_negotiation.cc:407] Connection header received
0809 20:23:12.274751 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:12.274758 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:12.274860 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:12.274918 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:23:12.275753 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:12.276582 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:12.278485 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:12.279030 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:12.279173 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:12.279650 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:23:12.279746 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:23:12.279849 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:12.280068 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 20:23:12.280145 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 20:23:12.280763 32174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:12.272905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47028 (local address 127.5.61.62:37621)
0809 20:23:12.273986 (+ 1081us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:12.273992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:12.274012 (+ 20us) server_negotiation.cc:407] Connection header received
0809 20:23:12.274126 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:12.274130 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:12.274271 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:12.274480 (+ 209us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:23:12.275288 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:12.276183 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:12.278124 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:12.278653 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:12.278774 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:12.279396 (+ 622us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:23:12.279831 (+ 435us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:23:12.279948 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:12.280343 (+ 395us) server_negotiation.cc:299] Negotiation successful
0809 20:23:12.280440 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":964,"thread_start_us":866,"threads_started":1}
I20260809 20:23:12.282635 31928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 20:23:12.274513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37346 (local address 127.5.61.61:44465)
0809 20:23:12.274671 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 20:23:12.274677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 20:23:12.275256 (+ 579us) server_negotiation.cc:407] Connection header received
0809 20:23:12.275509 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 20:23:12.275513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 20:23:12.275630 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 20:23:12.275733 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 20:23:12.277568 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:12.278601 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:12.280638 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 20:23:12.281197 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 20:23:12.281292 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 20:23:12.281918 (+ 626us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 20:23:12.282063 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 20:23:12.282175 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 20:23:12.282284 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 20:23:12.282375 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 20:23:12.286015 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 31759
I20260809 20:23:12.287616 31832 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:23:12.398026 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 31759
W20260809 20:23:12.422478 31952 connection.cc:535] server connection from 127.0.0.1:40424 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 20:23:12.422960 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 31844
I20260809 20:23:12.424110 31918 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:23:12.530879 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 31844
I20260809 20:23:12.553346 5364 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu pid 31931
I20260809 20:23:12.554569 32005 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 20:23:12.660351 5364 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7uf_Cz/build/asan/bin/kudu with pid 31931
2026-08-09T20:23:12Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4990 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10074 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (369581 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST